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EXECUTIVE SUMMARY 



Police Commissioner Raymond W. Kelly appointed the Crime Reporting Review 
Committee (the "Committee") 1 to determine whether the NYPD's internal- auditing and quality- 
control processes are sufficient to ensure the accuracy of the NYPD's crime-statistics reporting 
system, commonly known as "CompStat." The Committee's review focused primarily on audits 
conducted from 2009 through 201 1 and documentation and other information from 2009 through 
2012. This report describes the NYPD's crime-reporting and CompStat auditing processes. It 
sets forth the Committee's observations and conclusions concerning strengths and weaknesses in 
the CompStat auditing process, but it is not intended to be an audit of any CompStat statistics 
that have been reported through the CompStat process. Finally, the Committee has made a 
number of recommendations informally throughout the review process that have been favorably 
considered by the NYPD, and this report provides further specific recommendations for the 
NYPD to consider implementing in order to improve the CompStat processes. 



Robert Morvillo, an original member of the Committee appointed by Police Commissioner Kelly, 
contributed significantly to the work of the Committee that serves as the foundation of this report. 
Because of his untimely death on December 24, 201 1, before the drafting of this report, only the 
conclusions, opinions, and observations of the two remaining Committee members are included 
herein. The Committee also would like to thank Anirudh Bansal, a Cahill Gordon & Reindel LLP 
partner, and Victor Suthammanont, Samantha Walsh, and Victoria Rutherfurd, Cahill Gordon & 
Reindel LLP associates, for their extensive and invaluable assistance to the Committee. 
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BACKGROUND AND OVERVIEW 



Mission of the Committee 

The Commissioner appointed the Committee to conduct a review of the NYPD's 
CompStat auditing procedures and processes, focused particularly on whether the auditing 
processes were sufficient to ensure the accuracy of the NYPD's reported crime statistics. The 
Committee was not appointed to review or audit the crime statistics reported by the NYPD, but 
rather to assess the auditing processes that are now in place. The Committee, whose work has 
been done voluntarily, chose to review a broad spectrum of information that required a 
significant amount of time to distill. In addition, the Committee focused on certain audit 
procedures that, as the Committee was performing its review, were being updated and improved 
by the NYPD. This report attempts to summarize those updates and improvements and the 
resulting impact on the crime-reporting and auditing process. 



The Importance of Crime Recording and Reporting 

The recording and tracking of crime data serves both national and local interests. 
The NYPD, according to materials it provided to the Committee, uses crime statistics (as part of 
the CompStat program) to obtain intelligence about crime, analyze crime patterns, identify high- 
crime areas, develop strategies for combating or preventing crime, allocate resources, and 
measure the effectiveness of its efforts. 

When sent to the FBI as part of the Uniform Crime Reporting system, crime data 
may be used to compare crime statistics of major cities, evaluate anti-crime programs or grant 
programs, and conduct other various studies of crime and the criminal-justice system. For the 
public, published crime statistics help educate us about the level and types of crime risk we face. 4 
In addition, for some, crime statistics can provide a measure of the performance of our public 
servants. 5 



Key Terms and Concepts referenced in this Report are set forth in Appendix A. 

See, e.g., Uniform Crime Reporting Statistics: Their Proper Use available at 
http://www.fbi.gov/about-us/cjis/ucr/ucr-statistics-their-proper-use. 

Id. 

The FBI discourages using the UCR crime -reporting data as a measure of law-enforcement 
effectiveness. See A Word About UCR Data available at http://www.fbi.gov/about- 
us/cjis/ucr/word. 



Because of the many important interests served by the recording and tracking of 
crime statistics, it is imperative that such statistics be as transparent and accurate as possible. 
But transparency and accuracy may carry varying connotations to different constituencies. As 
relevant in the report, we will address the varying interests of law enforcement and the public. 

Methodology of the Committee's Review 

Information Gathering and Preliminary Assessment 

In conducting its examination of the NYPD's CompStat auditing procedures, the 
Committee followed a process designed to identify risks in the crime-reporting process and 
determine whether the controls imposed by the audit process appropriately address those risks. 
The first stage of that process was to gather information sufficient to enable the Committee to 
make preliminary assessments and further develop a workplan to guide its review. 

During the information-gathering stage, the Committee met with the 
Commissioner and Deputy Commissioner, Strategic Initiatives to discuss the Committee's 
mission and to receive general background on the NYPD's use and auditing of its crime 
statistics. The Committee also received more detailed briefings on the crime-reporting and 
auditing process from officers and officials from the NYPD's Quality Assurance Division 
("QAD") and Data Integrity Unit ("DIU"), the two units responsible for auditing the NYPD's 
crime-reporting program. 6 Those officials consisted of an Assistant Commissioner, a chief, an 
inspector, and two lieutenants. In addition, Committee members observed a CompStat meeting. 
Attendees at the CompStat meeting were not informed that Committee members were observing 
the session. 7 

In addition to meeting with NYPD officials, the Committee reviewed scholarly 
studies of the accuracy of CompStat reporting and the NYPD's audit process. During this 
process, the Committee met with John Eterno, Eli Silverman, Franklin Zimring, and Dennis 
Smith. Their studies are discussed or referenced as relevant within this report. The Committee 
spoke with representatives from some of the District Attorneys' Offices within New York City 
concerning their impressions of the accuracy of crime-reporting practices. The representatives of 
these District Attorneys' Offices provided the Committee with anecdotal information regarding 
crime reporting within their jurisdictions. The Committee also gathered and reviewed media 



Nothing in this report should be read as a criticism of QAD and DIU officers. The Committee, in 
fact, was impressed by the QAD and DIU officers' professional commitment and abilities. 

In addition to the observation of a CompStat meeting in 201 1, Committee members Mr. Kelley 
and Ms. McCarthy each had experience with CompStat meetings during their respective tenures 
as federal prosecutors in New York City. 
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reports concerning the accuracy of the CompStat reporting. Members of the public also 
contacted the Committee concerning their own crime-reporting anecdotes. 

The Committee also consulted with StoneTurn Group partner Jonny Frank, retired 
founder and head of PwC Fraud Risks & Controls and a former federal prosecutor and Yale 
faculty member with extensive experience in fraud risk assessments and investigations 
(including police misconduct). 

The Committee then utilized the information that it gathered to develop a detailed 
workplan to guide the in-depth review. Once the workplan was established, the in-depth review 
was conducted in two overlapping stages: (i) a review of the crime-reporting processes, risks, 
and controls, and (ii) a review of the effectiveness of the CompStat audit program's design, 
controls, and execution. 



Review of the Crime-Reporting Processes and CompStat Audit Program 

The Committee reviewed the crime-reporting process from initial intake to entry 
into the various computer systems comprising the CompStat system. The goal of this review was 
to identify risks and associated controls in the design of the reporting process prior to any audit 
by the NYPD. As set forth below, the Committee then reviewed whether the reporting process 
operated as designed, what additional risks arose from the actual operation of the process, and 
whether the controls were implemented as designed. The Committee then examined the NYPD's 
CompStat audit procedures to evaluate whether the design of the audit program sufficiently 
addressed the risks identified in the crime-reporting procedures. This review also included an 
assessment of the operating effectiveness of the NYPD's audit procedures. 

During its review, the Committee reviewed documents provided by the NYPD in 
response to requests from the Committee. The documentation the NYPD provided, from QAD, 
DIU, and the Office of Management, Analysis, and Planning ("OMAP") was extensive and 
included: training presentations, organizational charts, personnel descriptions, summary and 
individual audit reports of crime-classification audits conducted in 2009 and 2010, detailed 
QAD audit files from seven precincts (selected by the Committee), 9 proposals and files relating 
to new audit procedures, computer-system manuals, and crime-classification manuals. The 



2009 and 2010 were selected as the years in which to conduct sampling as they were, at the time 
the Committee began its work, the two most recently completed annual cycles. 

The Committee selected the precincts based on the volume of reportable offenses, choosing 
precincts with low, average, and high volumes of reported offenses. 
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Committee also examined audit files from the NYPD's newly implemented SPRINT audits. In 
addition, the Committee reviewed a report of QAD's investigation of allegations of downgrading 
and suppression in the 81 st Precinct (the "81 st Precinct Report") and other reports of QAD 
investigations concerning crime reporting. 11 The NYPD provided summaries of information 
where requested documents either did not exist or could not be provided because of the need to 
protect the confidentiality of investigations and/or personal information. The Committee also 
interviewed lead officers from QAD and DIU. 

In addition to the review of documents and its interviews with NYPD officials 
from QAD and DIU, the Committee visited four precincts to interview precinct staff involved in 
the crime-reporting and auditing processes. The purpose of the precinct visits was to gain a 
fuller understanding of the crime-reporting process within the precincts. The four precincts were 
selected by the NYPD to ensure that the Committee's inquiry would not disrupt on-going 
internal inquiries or audits and because personnel at those precincts agreed to meet with and 
assist the Committee. At the precincts, the Committee members typically interviewed the 
precinct commanders (usually inspectors or deputy inspectors), the precinct's Crime Analysis 
staff (including the sergeant in charge), and the desk officers (typically a lieutenant). In some 
precincts, other officers attended interviews — for example, additional staff from the Crime 
Analysis group or another senior precinct officer. The officers at the precincts impressed the 

12 

Committee as being forthcoming and candid during their interviews with the Committee. 
Limitations of the Committee's Review 

Although the NYPD's production of documents to the Committee was thorough 
and responsive, the Committee found its review of the QAD audit process limited by a lack of 
written protocols and audit plans. 

Overview of Crime-Reporting and CompStat Issues 

The NYPD's crime statistics are centered around the reported rates of seven major 
index crimes. Listed in order of seriousness, those crimes are murder, rape, robbery, felony 



As set forth in more detail herein, SPRINT audits are conducted to examine officers' responses to 
911 calls, which are referred to as "radio runs." The reports of such calls, including the 
disposition of the officers' response to such calls, are called SPRINT reports. 

See Graham Rayman, The NYPD Tapes Confirmed, VILLAGE VOICE, March 7, 2012. 

These interviews were productive and a vital element of the Committee's review. While broader 
access to more officers at more precincts selected by the Committee at random may have 
enhanced the quality of the review, the Committee decided not to pursue additional interviews in 
light of the information obtained during the initial round of precinct interviews. 
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assault, burglary, grand larceny, and grand larceny auto. According to NYPD statistics, 
between 1990 and 2010, index crime in New York City dropped 79.97%. Expressed in raw 
numbers, there were 527,257 reported index felonies in 1990 compared with 105,633 in 2010. In 
201 1, index crime rose by approximately 0.4% over 2010 levels, reportedly the first year-over- 
year increase since the implementation of CompStat in 1994. The issues raised by critics and 
scholars with respect to the accuracy of the NYPD's reporting of crime rates would not account 
fully for the reported drop in crime since 1990, and so, there appears to be little doubt that the 
crime rate in New York City is significantly lower now than it was in the 1990s. 

During this radical drop in major-crime rates for 20 straight years, the accuracy of 
the reported statistics has been questioned repeatedly. Anecdotal evidence, some reported in the 
press, of downgrading and suppression of crime reports has called into question, among some 
observers, the reliability of the reported statistics. 14 Reports in 2010, now substantiated, of the 
efforts of commanding officers in the 81 st Precinct to downgrade and suppress the reporting of 
crime in that precinct cast further doubt in some on the accuracy of the NYPD's numbers. 15 
Moreover, some studies point to the CompStat process as creating pressure to manipulate crime- 
reporting statistics. 1 Police unions also have alleged widespread manipulation of crime 
statistics. 17 

The thrust of the criticism is that management pressure to lower index crime 
creates incentives for precinct commanders to encourage the downgrading and suppression of 
complaint reports. This management pressure is alleged to be exacerbated by the CompStat 



13 



14 



15 



16 



17 



These crimes are referred to throughout this report as the "seven majors," "index crimes," and 
"index felonies." 

See, e.g., Rayman, supra note 11; Al Baker and Joseph Goldstein, Police Tactic: Keeping Crime 
Reports Off the Books, N.Y. TIMES, Dec. 30, 2011, at Al; Graham Rayman, Ray Kelly, They're 
Questioning Your Crime Stats, VILLAGE VOICE, February 7, 2010. 

See, e.g., Graham Rayman, The NYPD Tapes: Inside Bed-Stuy's 81st Precinct, VILLAGE VOICE, 
May 4, 2010; Rocco Parascandola, Brooklyn's 81st Precinct probed by NYPD for fudging stats; 
felonies allegedly marked as misdemeanors, NEW YORK DAILY NEWS, February 1, 2010. 

E.g., John A. Eterno and Eli B. Silverman, The NYPD's Compstat: compare statistics or compose 
statistics?, 12 INT'L J. OF POLICE SCIENCE & MANAGEMENT 426, 437 (2010). Professors John A. 
Eterno and Eli B. Silverman conducted a survey of retired captains and higher-ranking 
supervisors. The survey indicated that the CompStat process imposed on supervisory personnel 
substantial pressure to decrease index crime and that a significant percentage of respondents were 
aware of "unethical" changes to complaint reports. Id. 

The survey underlying the Eterno and Silverman scholarship has been criticized on a number of 
points, including the composition of the survey group and the phrasing of survey questions. See, 
e.g., William Bratton, Crime by the Numbers, N.Y. TIMES, Feb. 17, 2010. 

See Paul Moses, Corruption: It figures. NY police department's crime stats and the art of 
manipulation, VILLAGE VOICE, Mar. 22, 2005. 
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process, particularly the CompStat meetings, during which precinct commanders may be called 
upon, in front of their peers and high-ranking NYPD officials, to explain their job performance, 
which is measured in part by index crime rates in their commands. Precinct commanders 
purportedly induce, by threats or incentives, their subordinate lieutenants, sergeants, and patrol 
officers to downgrade crime or not to take or file complaint reports in order to avoid criticism at 
CompStat meetings. 

The response of the NYPD to these criticisms has been, in part, to point to the 
results of its internal audit process and its recent error rate of 1.5% in order to demonstrate that 
the reported crime statistics are accurate. 18 The NYPD also has cited scholarship that supports 
the accuracy of its reported numbers 19 and the adequacy of its audit program. 20 In addition, the 
New York State Comptroller audited the CompStat statistics in 2000, concluding that the 
statistics were supported by the underlying complaint reports. 21 



See, e.g., Baker and Goldstein, supra note 14 (quoting NYPD spokesman as referring to internal- 
audit error rate of 1.5%). This report further discusses the NYPD's calculation of the 1.5% error 
rate at pp. 49-52. 

See Franklin E. Zimring, THE CITY THAT BECAME SAFE: NEW YORK AND THE FUTURE OF CRIME 
CONTROL (Oxford Univ. Press 2011); Patrick A. Langan & Matthew R. Durose, The Remarkable 
Drop in Crime in New York City (2004). The Committee reviewed Mr. Zimring' s book, as well 
as an early excerpt he provided describing his means of confirming the NYPD's reported 
statistics. We will address the reporting and scholarship further as necessary in later sections of 
this report. 

Dennis C. Smith and Robert Purtell, Occasional Paper, Managing Crime Counts: An Assessment 
of the Quality Control of the NYPD Crime Data (2006). 

State of New York Office of the State Comptroller, Division of Management Audit and State 
Financial Services, Report No. 2000-N-8, New York City Police Department: Accuracy of Crime 
Statistics (2000). 
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THE CRIME REPORTING PROCESS 



Overview of the Process 
The Initial Report 

22 

Complaint reports typically originate in one of three ways: a 91 1 call, a precinct 
walk-in, or a pick-up (in which an officer encounters a complainant while on patrol). Responses 
to 91 1 calls, also known as "radio runs," typically involve patrol officers. If the crime reported 
to the 91 1 dispatcher appears to be a felony or if the responding officers discover that a felony 
has been committed, a supervising officer (e.g., a sergeant or lieutenant) may report to the scene 
as well. 23 

Radio runs, walk-ins to precincts, and pick-ups by officers on patrol may have 
multiple dispositions. Responding officers may (i) find the complaint "unfounded" (i.e., no 
crime was committed), (ii) find the complainant is uncooperative or unavailable (e.g., police 
respond to a scene to find no one present), or (iii) take a complaint report. 

In the case of radio runs, if a crime has been committed, the initial complaint 
report is typically taken at the scene. For walk-ins, the complaint reports are usually taken at the 
precinct. On occasion, a walk-in complainant reporting certain location crimes, such as a 
burglary or robbery, is instructed to return to the scene of the crime with patrol officers in order 
to take the complaint report at the location and to facilitate investigation of the crime or enable 
canvassing by the officers. 

However the complaint originates, it is recorded initially on a Complaint Report 
Worksheet (also known as a "61"), a detailed form that guides the officer in recording 
information necessary for, among other purposes, proper classification of the crime. The original 
complaint report, known as the "scratch" report, is taken by hand and might reflect scratch-outs 
and edits from the officer completing the report. The scratch copy is filled out according to 



When officers respond to a 91 1 call, even if there is no report to take, the incident is recorded in 
the responding officers' memo books. In addition, a record of the 911 dispatch and report back to 
the dispatcher, known as a SPRINT report, is generated. 

This is not a policy of the NYPD, although it appears to be a widespread practice among 
precincts, including those the Committee visited. The practices of certain precincts may also vary 
with respect to which felonies supervising officers respond. 
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procedures set forth in the NYPD's Crime Complaint Reporting System Reference Guide (the 
"Reference Guide"). 24 

In cases not involving firsthand observation of a crime by the officer, the report is 
based solely on the complainant's account of events. In cases where a complainant reports lost 
or stolen property, the complainant reviews and signs a Lost/Stolen Property Form listing the lost 
or stolen property and the value of such property. Thus, in most instances, the complainant 
assigns a value to lost or stolen property. 25 

NYPD policy requires the responding officer or the precinct in which the 
complaint is made to take the complaint report, even if the crime occurred in another precinct's 
jurisdiction. The rule is summed up in the axiom — as quoted to the Committee in various 
interviews: "Refer the complaint, not the complainant." In such situations, the complaint report 
is taken by one precinct and ultimately, for both investigative and reporting purposes, transferred 
to the precinct in which the crime occurred. In instances where the location of the crime is 
unknown — e.g., a complainant traveling between an office in one jurisdiction and home in 
another discovers her wallet missing after transit between the two jurisdictions — the crime is 
recorded in the location of its discovery. 



Crime Classification 

Part of the report-taking process is the classification of the crime or crimes 
reported, which is performed initially by the officer preparing the complaint report. The 
Reference Guide contains detailed guidelines to assist officers in classifying crimes, and it is 
constantly being updated as laws and internal NYPD procedures are revised and implemented. 
A complaint report must classify the crime as the most serious possible crime and as the highest 
degree of that crime as permitted by the complainant's description, regardless of other 
considerations such as what crime a district attorney might charge. The classification of crimes 
follows the hierarchy of the seven major felonies, or index crimes: murder (including non- 
negligent manslaughter), rape, robbery, felony assault, burglary, grand larceny, and grand 
larceny auto. While multiple crimes may be recorded in the complaint report, only the most 

The Reference Guide is revised and published by DIU and OMAP. It is revised when significant 
changes in the Penal Law require clarification or when the audits suggest there is an issue 
requiring an explanation or increased emphasis. The most recent revision was in June 201 1. 

The Lost-Stolen Property Form was revised in 2005 to allow the complainant to designate items 
as lost or stolen and the property value. At that time, the policy to use the complainant's 
valuation as the recorded valuation took effect. This policy was emphasized in the June 201 1 
revision of the Reference Guide. 

This policy, as well as others governing complaint reporting, was reiterated in an Operations 
Order circulated in January 2012 to all commands. 
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serious crime is officially counted for purposes of crime statistics. Finally, attempted crimes 
are classified (for reporting purposes) as if the crime were completed, except attempted murder, 
which is classified as felony assault. 

In addition to the major-crime and misdemeanor categories reported publicly, the 
NYPD tracks and internally reports other categories of felonies and misdemeanors, as well as 
lost-property complaints, for its own strategy initiatives. The NYPD treats "lost property" as a 
non-crime investigation classification for internal statistical purposes. Lost property is missing 
property that is not classified as stolen. 



"Sign-off of Complaint Reports 

After a complaint is taken, the scratch copy is given to the precinct's desk officer, 
who must sign it. After the scratch copy is signed, it is taken to the "124 room," where a clerk, 
typically a civilian, enters the data from the scratch copy into the NYPD's Omniform system, 
which is used to record complaint reports as well as arrest reports. Occasionally, particularly 
when there has been an arrest, the officer completing the scratch copy may enter the information 
into Omniform herself. Omniform automatically generates tracking numbers for complaint 
reports and arrest reports. 

Once the complaint report is entered into Omniform, and a complaint number is 
generated, the report is in "edit mode." In this mode, the complaint report can be edited by any 
user, such as the 124-room clerk or a supervisor. The desk officer's sign-off is required in order 
to complete entry of the report. The 124-room clerk prints from Omniform a computer- 
generated version of the complaint report and brings the printout and the scratch copy to the desk 
officer. The Omniform version contains only those fields that have been completed and is a 
more compact document than the scratch copy. At this point, the Omniform complaint report is 



Consider, for example, a situation where a suspect punches a complainant, causing injury, and 
takes the complainant's wallet which contains $1200. Under the Penal Law, the conduct might 
constitute the crimes of robbery, assault, or grand larceny. For crime -reporting purposes, the 
responding officer must report the crime as a robbery in the second degree, the highest level of 
the possible offenses. 

This methodology comports with the FBI's Uniform Crime Reporting rules. See FBI, Uniform 
Crime Reporting Handbook, p. 10, available at http://www.fbi.gov/about-us/cjis/ucr/additional- 
ucr-publications/ucr_handbook.pdf#page= 17. 

Some arrests, such as those for certain drug-possession crimes, do not generate a complaint report 
but do generate an arrest report. Omniform gives users the option of inputting arrest data without 
generating a new complaint. 

The aging of complaint reports in edit mode is tracked. NYPD policy requires that such reports 
be finalized within 24 hours. 
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the official record. The desk officer compares the scratch copy against the Omniform report and 
signs off on the complaint report in Omniform. Among the considerations for sign-off, as set 
forth below, is the proper classification of the reported crime. 

Sign-off is a "Supervisory Function" in Omniform and is available only to 
supervising officers, e.g., sergeants and lieutenants. Similarly, complaints and arrests may be 
voided only by officers with access to the Supervisory Functions in Omniform. After the report 
is signed-off at the precinct level, it appears on CompStat reports that are distributed up through 
the chain of command. 

Precinct-level Review and Amendment 

Although the sign-off completes the crime-reporting process for purposes of the 
inclusion of a complaint report in reported crime statistics, two other precinct-level reviews 
occur: Crime Analysis and commander reviews. It is not uncommon for these regular reviews to 
result in amendments to the complaint reports that affect the reported statistics, so we address 
them here as part of the normal crime-reporting process. 

Within each precinct, the Crime Analysis team (typically comprised of a sergeant 
and several police officers) reviews the complaint reports entered the previous day — particularly, 
but not limited to, the seven major index crimes — and also has access to the complaint reports 
that have not yet been completed and are in edit mode. Crime Analysis teams perform a number 
of functions, but with respect to entry of the complaint reports, they serve as an extra level of 
review for many reports. They review the scratch copy along with the Omniform copy of a 
complaint report. If it appears that a complaint has been misclassified, the Crime Analysis team 
may follow up with the reporting officer or the desk officer. Occasionally, the Crime Analysis 
team will follow up on complaint reports by calling the complainant in an effort to clarify the 
facts of the crime to ensure that it has been classified properly. 

Commanding officers also review complaints entered into Omniform by their 
commands. The Committee spoke to the commanding officers of four precincts. Each said that 
they (or, in high- volume commands, they and their executive officers) review all or nearly all of 
the felony complaint reports entered in their precincts on a daily basis. 

Either of these levels of review may result in an amendment to a complaint report. 
The Crime Analysis team or the commanding officers may disagree with the reporting officer's 
classification in light of their experience, or may identify an undeveloped fact or issue that, when 
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resolved, causes changes to the classification of the crime. In such cases, Omniform allows 
amendment of the complaint reports as discussed below. 30 

Amendments are made using a complaint follow-up form (known as a "DD-5") 
and by entries into Omniform (also referred to as a "DD-5")- DD-5s are further divided by color 
depending on the type of change being made to the complaint report. Some of the changes may 
affect CompStat reporting ("blue DD-5s") — for example, officers may upgrade a crime after the 
victim of a larceny discovers additional property was stolen — while others may not ("pink DD- 
5 s") — for example, detectives can add details that aid the investigation of the crime but do not 
change its classification. Changes to a filed complaint report require supervisory sign-off before 

3 1 

they can take effect. Omniform tracks when and by whom complaints have been updated. In 
addition, changes are manually recorded in a log at the precinct, which itself requires multiple 
levels of sign-off. Omniform limits access to complaint reports, and officers generally have 
access only to the complaint reports filed within their own commands. 

The chart on the following page illustrates the crime-reporting process. 



In addition to these regular reviews, it is possible for the detective squad to change the 
classification of a crime as a result of its investigation, e.g., label a crime "unfounded." In that 
event, the appropriate commanding officer must explain the change in classification by 
completing a form known as a "DD-5." 

In addition, changes affecting classification require review and approval of DIU as described later 
in this report. 

In addition to the precinct-level procedures discussed, there are other reviews that touch upon the 
crime-reporting process. For example, officers' memo books are reviewed by patrol supervisors 
during each tour. In turn, the Integrity Control Officer reviews and signs the sergeants' memo 
books, which should reflect the inspection of the patrol officers' books. 
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NYPD Crime Reporting Process 



1. Complaint is Made 



911 call 



Walk-In 



Pick-Up 



2. Officer Completes a Scratch Report 



If appropriate Complainant completes Lost/Stolen 
Property Form 



3. Precinct Reviews and Files Report 



Desk Officer Signs-off (scratch report) 



124 Room enters complaint into Omniform 



Desk Officer Signs-off (Omniform version) 



Complaint is filed in Omniform 



4. Filed Complaint is Further Reviewed 



Commander Review 



Crime Analysis 
Team Review 



5. If Necessary, Amendments Are Made 



DD-5 form created with amendment 



Changes are recorded in Precinct Log 



Supervisory Sign-off 



Complaint is updated in Omniform 
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The Crime-Reporting Process: Limitations and Risks 



There are risks to and limitations on the accuracy of the crime statistics tracked by 
the NYPD. In order to evaluate the efficacy of the NYPD's auditing program, these risks and 
limitations must be identified in order to determine whether the existing controls (in the crime- 
reporting and auditing processes) address them. 



Limitations of Crime Reporting 

Strict Interpretation Bias. The NYPD's procedures, as well as the Uniform 
Crime Reporting procedures, require that complaint reports be classified in the most serious 
category of crime described by the complainant regardless of other factors, such as ability to 
actually prove the case or which crime is charged on arrest. This rule can yield what has been 
termed a "strict interpretation bias" in favor of classifying an incident as the most serious 
possible category of crime. This means that crime reporting generally reflects the most serious 
view of crime, as opposed to a more nuanced and a potentially more realistic view. In addition, 
the strict-interpretation bias also affects error rates. Because of this bias, the theory is that, as 
more individuals review an incident recorded on a complaint report, there is an increased 
probability that a reviewer may believe that a complaint report should be upgraded. 34 Because of 
this strict-interpretation bias, most errors affecting the classification of a crime should tend to 
require upgrading. Lateral or downgrading changes should occur less frequently. 

Unreported Crime. Because crime statistics rely upon the reporting of crime, a 
limitation on their accuracy is unreported crime. Unreported crimes are those crimes not 
reported to the police — a phenomenon distinguished from downgrading and suppression. Crimes 
may go unreported for any number of reasons personal to the victim: fear, a desire to avoid 
contact with the police, embarrassment, or even a lack of time. According to the 2010 National 

Of 

Crime Victimization Survey, 50% of violent crime and 60% of property crime was unreported. 

Complainants' Limitations. Similar to unreported crime, the limitations of 
complainants may undermine the accuracy of reported crime. For example, complainants may 
be unable to remember or may omit key facts, lie to police officers, or — particularly in New 



Smith and Purtell, see supra note 20, at 19. 

For example, one distinction between third-degree assault (a misdemeanor) and second-degree 
assault (a felony) is whether a "serious" physical injury occurs. Compare N.Y. Pen. L. § 
120.00(1) with § 120.05(1). In an instance where an injury is as likely to be considered "serious" 
as not, each level of review introduces a chance that the reviewer will consider the injury 
"serious" and, therefore, a felony assault. As Smith and Purtell note, see supra note 20, this will 
tend to increase the weight of reported crime. 

Available at http://bjs.ojp.usdoj.gov/content/pub/pdf/cvlO.pdf. 
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York City with its large transient population of tourists and business travelers — lack the time or 
communication skills necessary to give a complete report, all of which may affect the accuracy 
of the ultimate crime classification. 36 

Subjective Nature of Classifications. Classifying myriad, complex, real-life 
situations within the rigid confines of the text of the New York Penal Law is a subjective 
exercise dependent on the judgment of the individuals involved in taking the complaint report. 
While a crime classification may encompass a wide range of prohibited conduct, classifying 
crimes requires that like circumstances be treated similarly. But experienced judges, 
prosecutors, and police officers may all in good faith disagree as to what crime has been 
committed (or whether one has been committed). Two examples highlight this issue: 

• A complainant reports that a first-floor window of his home was broken 
by an unknown perpetrator. Should this be classified as criminal mischief 
(a non-index crime) or an attempted burglary (an index crime)? 

• A complainant reports that she forgot her backpack (containing a credit 
card) on a park bench and, upon returning an hour later, discovered the 
bag missing. Should this be classified as a grand larceny (i.e., the bag was 
taken with the intent to deprive the complainant of the property) or lost 
property (i.e., the bag was taken by a Parks Department employee and 
deposited in the lost-and-found)? 

The NYPD has over 30,000 police officers evaluating, judging, and classifying 
hundreds of thousands of events like these each year. For risk-assessment purposes, this 
subjective variation is an effect related to, but distinct from, human-error and downgrading risks. 
The magnitude of variation in reported crime that is introduced by the element of subjectivity — 
e.g., whether the effect is uniform year-by-year and what bias it introduces — is unknown. But 
considered in light of the decreasing yearly differences in crime rates (for example, there was a 
0.4% decrease in reported grand larcenies between 2010 and 201 1), this effect is likely becoming 
material because, as the number of reported crimes decreases, even slight variances are 
significant. 

Arbitrariness. In addition to subjective variation, the accuracy of crime statistics 
is affected by arbitrariness. Arbitrariness may be introduced into the system by establishing non- 
discretionary guidelines to promote uniformity among classifications. For example, in order to 
prevent officers from improperly reducing the value of stolen property to avoid grand-larceny 
classifications, the rule as of June 2005 is that, in virtually all cases, the value of stolen items as 



These examples were drawn from the Committee's precinct interviews. 
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reported by the complainant determines the value for purposes of crime-classification. So, for 
example, if two of the same two-year-old laptops (purchased for $1500) are stolen, and one 
complainant values the laptop at the purchase price ($1500) and the other accounts for 

38 

depreciation (valuing it at $800), the same stolen item is a grand larceny in the former instance 
(the $1,500 value) and a petit larceny in the latter (the $800 value). In other words, this rule 
substitutes potentially arbitrary valuations by complainants (an accuracy issue) for potentially 
arbitrary valuations by police officers (an accuracy and integrity issue). Another example of a 
rule promoting uniformity leading to arbitrariness relates to stolen or lost property involving a 
credit or ATM card. Regardless of credit limits or cash balances, if such a card is stolen, it is 
treated as a grand larceny for reporting purposes. 

Changing Standards. Substantive standards of what conduct constitutes which 
crimes, the procedures of how complaint reports are taken and entered into the system, and law- 
enforcement strategies (such as focusing on particular categories of crime) can affect the 
accuracy of crime statistics, particularly as a comparative matter. 

A substantive example is the addition of strangulation to the Penal Law. 40 The 
Penal Law was revised so that strangulations that were classified as harassments or misdemeanor 
assaults before the amendment, due to the lack of apparent injury, are now classified as felony 

The Reference Guide provides detailed rules concerning how to determine the value of lost or 
stolen property. In addition, a high-level QAD member informed the Committee that Crime 
Analysis staff may check certain acceptable Web sites to determine the true value of lost or stolen 
property only in instances when the value stated by the complainant is clearly incorrect and may 
affect the classification of the crime. 

In at least one precinct visited by the Committee, a Crime Analysis sergeant said that she 
performed independent research to determine the value of lost or stolen property despite the 
valuation attributed to such goods by a complainant. In the context in which the statement was 
made, it appeared that the research and verification diverged from the policy set forth in the 
Reference Guide. Other Crime Analysis sergeants denied performing such independent research. 

As further discussed in this Report, this practice is an example of a subjective decision being 
made by Crime Analysis staff that undermines the purpose of the Lost/Stolen Property Form 
being completed by the complainant and potentially affects the grand-larceny statistics. 

We note that valuation by the complainant is a subjective factor (similar to the broken-window 
example above), but insofar as it is outcome determinative without respect to — or in defiance 
of — other objective factors (e.g., compared with the broken-window example, where objective 
factors such as the extent and location of the break, etc., may be determinative of the subjective 
assessment), we distinguish it as arbitrariness. 

The rule, while not preventing arbitrary valuations, does promote (a) greater integrity among 
police officers by decreasing their ability to affect a fact that might alter crime classification and 
(b) less adversarial interactions with complainants, whose valuations are not supposed to be 
challenged. 

40 NY Penal Law §§ 121.11 and 121.12(2010). 
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assaults. Accordingly, the number of felony assaults may rise because the conduct that qualifies 
as such a crime has been broadened. 

There are also examples of procedural changes that may affect the classification 
of offenses. The Committee was told that some of the decrease in reported auto thefts may be 
attributable to new procedures designed to attempt to locate a missing automobile prior to taking 
a complaint report. As described to the Committee, the practice of the NYPD approximately 20 
years ago was to take a complaint report if a complainant claimed their automobile was missing. 
Current practice is to take a complainant back to the scene to look for the automobile — although 
the practice was designed to screen for instances of attempted insurance fraud, apparently it is 
not uncommon for a complainant to have forgotten that they parked their car in a different 
location. 41 Similarly, as previously described, the procedure requiring acceptance of a 
complainant's valuation in stolen-property complaints differs from prior practice. Therefore, 
comparison of recently reported larcenies is not an apples-to-apples comparison to larcenies 
reported prior to these changes. 

Risks in the Crime-Reporting Process 

Human Error. In the crime-reporting process, the following individuals 
typically handle every complaint report: the responding officer, a supervisor (in the case of 
certain felonies), the desk officer, the 124-room clerk, and the desk officer again. In addition, 
the complaint report is reviewed after the sign-off into Omniform by the Crime Analysis staff 
and, in many cases, the commanding officer. Each of the individuals handling the complaint 
reports may introduce errors affecting the classification of the complaint report — or whether a 
report is taken at all. For example, a responding officer taking an assault complaint may fail to 
record that the assault caused an injury and classify the assault as misdemeanor assault, when in 
fact it should be a felony assault. The error would persist throughout the subsequent review 
because the description in the complaint report would match the classification. 

Human error may be an effect of lack of training, inexperience, inattentiveness, 
and work overload due to a high volume of reports, see infra note 120, among other things. We 
distinguish unintentional errors (misfeasance) from intentional manipulation of the complaint- 
reporting process (malfeasance). 

Manipulation (Downgrading & Suppression). Manipulation of crime statistics 
can take the form of either downgrading or suppression. Downgrading is taking a complaint 

41 To be clear, auto theft has decreased (e.g., 146,925 in 1990 vs. 10,307 in 2010). It is unlikely that 
there were 136,000 people per year attempting insurance fraud or forgetting where they parked 
their cars and then attempting to file complaint reports. The issue is addressed merely to illustrate 
that a change in procedures can affect reported crime. 
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report, but intentionally misclassifying the crime as one less serious than described. Suppression 
is the failure to take a complaint report, whether by discouraging or preventing a complainant 
from filing one or failing to enter a scratch copy into Omniform. Press reports have described 
instances of each, and the Committee has heard anecdotal evidence of each as well. In addition, 
the 81 st Precinct Report describes instances of downgrading and suppression. 

According to some published accounts, manipulation is the product of 

42 

accountability pressures. It is important to recognize for purposes of risk assessment that this 
accountability pressure is not limited to precinct commanders. As noted, the Commissioner and 
top-level NYPD officers are also measured in part by movement in reported-crime statistics. As 
such, there is a tension between those officials' desire to see continued crime reduction and their 
need for accurate statistics to develop and implement policing strategies, deploy officers, and 
otherwise measure accountability throughout the police force. 

The theory — supported by real-life examples (most recently in the 81 st 
Precinct 43 ) — is that precinct commanders manipulate statistics through downgrading and 
suppression. Because accountability is based on reported crime (i.e., the index crimes), one way 
to "reduce crime" is to reduce reports of index crime — either by classifying complaint reports in 
a non-index-crime category or by suppressing reports. In this way, even if crime reports are flat 
or increasing, they can be made to appear to be decreasing. 44 

To be sure, from the Commissioner to precinct commanders, performance is — at 
least in part — measured by the ability to "reduce crime." Nevertheless, the Committee has been 
informed that no commander has lost his or her command solely due to a rise in crime. Rather, 
performance is measured by other factors, including whether a precinct commander appears to 
have a plan to address any such rise in crime. Indeed, those commanders interviewed by the 
Committee indicated that a rise in crime rates in their jurisdiction, particularly in index crimes, 
may not be a cause for an unpleasant experience at a CompStat review. Instead, the pressure 
would be on them to devise a plan that could address the rise. One commander, however, 



See, e.g., Eterno and Silverman, supra note 16. 

In the 81 st Precinct, Police Officer Adrian Schoolcraft recorded hundreds of hours of tapes of 
precinct roll calls and other conversations at the precinct. The recordings revealed precinct 
supervisors exerting significant pressure on patrol officers not only to meet quotas on activity 
such as traffic tickets, but also to manipulate or not take crime reports. See supra note 15. 
Officer Schoolcraft's allegations were largely substantiated by QAD's investigation of the matter 
as recorded in the 81 st Precinct Report. 

Nevertheless, decreases in certain crimes such as homicide and grand larceny (auto) are 
independently supported by coroner's reports and insurance claims. See Zimring, supra note 19. 
Despite the debate about whether the extent of the decrease is accurate, none of the scholars 
interviewed by the Committee contested the fact that overall crime in New York City was 
substantially lower now as compared to the early 1990s. 
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indicated that he focuses his attention predominantly on index crime (which are the "major" 
crimes), the CompStat statistics, and crime reduction (as measured against the prior-year 
numbers). 45 

One precinct-level officer told the Committee that there also may be pressure to 
treat classifications differently during different times of the year or even to upgrade crimes. For 
example, towards the end of the year, a precinct commander who knows that he will beat his 
prior-year crime totals may stop challenging index-crime classifications in an effort to control 
the target for the following year and avoid becoming, as the officer put it, "a victim of his own 
success." In other words, at least in the view of this officer, it may be better for a commander to 
show steady, but modest, decreases rather than a drastic, but unrepeatable, decrease. Obviously, 
this is purely anecdotal, and the Committee cannot say whether this view is shared by other 
officers. 

At the patrol officer and immediate supervisor levels, the incentives to manipulate 
statistics are different. The theory, again bolstered by the reports in the 81 st Precinct case, 46 is 
that precinct commanders exert pressure on their immediate subordinates to take actions that 
serve to manipulate crime statistics. The precinct commanders interviewed by the Committee 
denied that patrol officers or their superiors had any incentive to follow such orders, a sentiment 
also expressed by other officers who spoke with the Committee. Despite these statements, the 
fact of an order is its own incentive in a hierarchical organization, and the Committee is of the 
view that the risk exists and, as indicated, there have been substantiated reports of manipulation 
in the past. 47 



In addition, each commander that the Committee visited had in his office a board of reported 
index crimes tracking the current-year numbers against the prior-year numbers for the particular 
precinct. 

See infra note 121. In addition to the 81 st Precinct case, there was another reported instance 
where a commanding officer clearly used his influence to pressure a sergeant to downgrade a 
report and engaged in retaliatory actions when it was clear that QAD was investigating the 
incident. QAD thoroughly investigated the matter (including the retaliation) and the commanding 
officer lost his command. There were several older instances in which commanding or high-level 
officers were disciplined for crime-reporting manipulation as well. 

In most other recent investigations, complaints against commanding officers were not 
substantiated by QAD, even in cases where large numbers of supervising officers were implicated 
in downgrading and/or suppression. An explanation for this result may be the difficulties in 
investigating and proving allegations as described in the section concerning QAD investigations. 

Indeed, in the investigative reports reviewed by the Committee, the vast majority of officers 
implicated in downgrading and suppression were patrol officers, sergeants, and lieutenants. In 
one instance, a group of detectives was implicated. Nevertheless, officers rarely admitted to 
facing pressure to downgrade or suppress complaint reports — even when admitting to having 
done so. 
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The Committee also noted areas in the NYPD's crime-reporting process that are 
particularly susceptible to manipulation. For example, the current reporting system does not 
allow a scratch report to be entered into Omniform prior to sign-off by the desk officer, nor is the 
report fully entered into Omniform prior to a second cross-check by the desk officer. At this 
stage, a DD-5 is not necessary to effect a change in the complaint report's classification and, 
though Omniform tracks when and by which user a complaint was accessed, it does not track 
what information, if any, was changed. Therefore, a desk officer exerts considerable control 
over the complaint report at its most vulnerable control point (i.e., prior to the point when 
changes are electronically tracked and subject to review by DIU). 48 

Similarly, while the policy of having supervising officers (e.g., lieutenants and 
sergeants) respond to scenes of felonies advances the purpose of having a senior, experienced 
officer to assist and guide patrol officers in taking an accurate and complete report, the policy 
also may afford an opportunity for those supervisors to significantly affect the outcome of the 
classification at a point where the controls on suppression and downgrading are among the 
weakest — a risk not unlike that presented in the 81 st Precinct case where supervising officers 
reportedly exerted pressure on subordinate officers to downgrade or suppress complaint reports. 

Finally, some allegations of complaint-report manipulation arise from 
complainants' requests for copies of their own complaint reports or complaint-report numbers. 49 
In responding to these requests, NYPD personnel occasionally have discovered that a complaint 
report for an incident does not exist or fails to accurately reflect the complaint. Although the 
NYPD keeps records of when a complainant requests a copy of a complaint report from the 
Central Records Division, these records would not capture data about complainants requesting 
information directly from a command. This information could be a useful tool for QAD to 
uncover a pattern within a command of officers being unable to locate incidents in Omniform 
after a query by a complainant. Knowledge that such information was being maintained would 
also have a deterrent effect on officers potentially seeking to suppress complaint reports. 

Controls in the Crime-Reporting Process 

Multiple Levels of Review. Multiple levels of review in the precinct are 
designed to promote more uniform subjective decisions and to mitigate the risks of human error 



The risk presented by this weakness is illustrated by the 81 st Precinct Report. QAD's findings 
appear to reflect that certain desk officers were involved in multiple instances of alleged 
suppression or downgrading. 

This was a pattern apparent in the IAB referrals to QAD as well as reported in the media. See, 
e.g., Baker and Goldstein, supra note 14 ("It is not unusual for detectives, who handle telephone 
calls from victims inquiring about the status of their cases, to learn that no paperwork exists.") 
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and, to a lesser extent, manipulation. The review by more experienced supervisors, Crime 
Analysis staff, and precinct commanders provides opportunities for discussion, correction, and 
ultimately more accurate classifications. Precinct staff told the Committee that, during the initial 
review process, they would follow up with responding officers if the classification of a crime 
appeared to be inadequately supported or the complaint report lacked sufficient detail. 50 

Moreover, routine data-entry mistakes are more likely to be caught with multiple 
levels of review. For example, if a complaint report lists the value of a stolen necklace at $200, 
but the 124-clerk entering the property into Omniform accidentally omits a zero (making the 
value $20), subsequent reviews by the desk officer, Crime Analysis staff, and commanding 
officers increase the chances of discovering the error. 

In addition, multiple levels of review provide a control on intentional malfeasance 
and may uncover attempts to intentionally downgrade crimes. Knowledge that other officers 
review complaint reports and are able to detect downgrading may discourage attempts to 
downgrade. If the precinct-level review process is compromised, however, multiple-level review 
at the precinct level would not be an adequate control for malfeasance, particularly because, as 
discussed previously and as recent reports illustrate, incentives to downgrade crime exist at the 
highest levels of review within the precinct. 

Precinct-level Audits. In addition to the QAD and DIU audit processes 
described later in this report, there are audits conducted at the precinct level, typically by the 
Integrity Control Officer ("ICO"), of SPRINT reports and dispositions. These audits involve the 
ICO reviewing a required number of SPRINT reports for a given week within the month and 
obtaining and checking the corresponding complaint report. Formerly, the ICO would only 
attempt to locate a corresponding complaint report if the disposition indicated that a complaint 
report was prepared, but now all dispositions are audited. These precinct-level audits are not 
designed to be as robust as the QAD SPRINT audits described later in this report. They do, 
however, provide an additional control on suppression or error at the precinct level. 

Amendment Procedures. After a complaint report is signed off into Omniform, 
it can only be amended through the DD-5 process. As described earlier, the amendment process 
(for those changes affecting crime classification) is tracked through the use of forms (which 
themselves require multiple levels of approval), logs at the precinct level (which also require 



For example, a complaint report might be classified as an assault in the third degree 
(misdemeanor assault), with a supporting detail section stating only: "complainant reported that 
unknown perpetrator punched her." The reviewing desk officer or Crime Analysis staff might 
then follow up with the reporting officer and/or complainant to inquire whether the complainant 
sustained an injury or whether any property was stolen, details that would affect the classification 
of the crime and provide critical details for investigative and crime analysis purposes. 
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multiple levels of approval), and approval by DIU. These procedures, particularly the precinct- 
independent review by DIU, present significant hurdles to post-entry manipulation. 

Training. The NYPD provides regular training to its officers, and also provides 
additional training for newly promoted officers. QAD performs promotional training, ad hoc 
training of field personnel, and "conferrals" with precinct commanders. Training incorporates 
changes in the Penal Law, policies, and procedures that may affect crime classification. 
Additionally, when issues are detected by QAD audits, additional training may be provided to 
address these issues (as will be described further). Training ideally mitigates the risk of error by 
promoting more uniform subjective judgments, and ideally builds a culture in which 
manipulation is not tolerated at the precinct level. 

Hierarchy and Culture. Commanding officers and officials can set a "tone at 
the top" with significant ability to inculcate that tone throughout the institution. Therefore, 
commanding officers can promote integrity throughout the organization. The precincts visited 
by the Committee demonstrated the positive effect of commanders who demanded accuracy in 
the crime-reporting process. In addition, QAD personnel interviewed by the Committee all 
demonstrated deep commitment to their mission. 

Similarly, an institutional culture of integrity can discourage downgrading and 
suppression. Precinct-level officers indicated to the Committee that the culture surrounding 
complaint-reporting had changed from "what it had been." Other changes can mitigate risks, 
such as accountability pressure. The Commissioner's staff indicated that changes in the 
CompStat meeting process made such meetings less adversarial, which might have the effect of 
decreasing accountability pressure. Each precinct commander the Committee interviewed said 
that CompStat meetings added no additional performance pressure. 51 

Summary of Conclusions 

In summary, the Committee's assessment of the process-related risks is that 
human error and manipulation are the most substantial risks in the complaint-reporting process. 
With respect to human error, multiple levels of review at the precinct level provide satisfactory 
control to ensure that reported crime is sufficiently accurate for CompStat purposes. 

Process-level controls, however, do not sufficiently address the risk of intentional 
manipulation. First, as noted above, intentional manipulation assumes that at least one level of 
control — an officer in the process — is compromised. Second, downgrading and suppression can 

The opinions expressed by the current precinct commanders (selected by NYPD headquarters) 
stand in marked contrast to the opinions of anonymous and retired survey respondents obtained 
by Eterno and Silverman. See supra note 16. 
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be difficult to detect because, prior to the entry of a complaint report into the Omniform system, 
nearly all elements of reporting are controlled by the individuals from whom the system is at risk 
of manipulation. Therefore, in a scenario assuming a high degree of risk — i.e., an officer who 
wishes to manipulate crime reporting — few other procedures are in place that control the various 
avenues of potential manipulation, particularly prior to the entry of the complaint report into the 
Omniform system. 
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THE COMPSTAT AUDIT PROCESS 



Overview of the CompStat Audits 

The complaint-reporting system is audited by two dedicated teams — QAD and 
DIU. QAD conducts semiannual audits of each of the NYPD's 97 commands, comprised of 76 
Patrol precincts, 12 Transit Districts, and 9 Police Service Areas, testing the accuracy of each 
command's complaint reports and compliance with other regulations such as, for example, the 
posting of certain notices. DIU audits a random sample of 25,000 complaint reports across all 97 
commands for data inconsistencies. DIU also audits all the change reports affecting 
classifications (blue DD-5s) within Omniform. 



Quality Assurance Division 

QAD is comprised of two teams that review and audit complaint reports. The 
first team, the Complaint Assessment Team, consists of a lieutenant and 14 sergeants, divided 
into smaller groups (typically three sergeants) and assigned different precincts or commands to 
audit. The second team, the Confidential Investigations Team, consists of five members 
(supervised by a lieutenant) who conduct investigations of malfeasance in the complaint- 
reporting system. According to QAD personnel, both teams have grown in recent years to 
accommodate increased workloads. 

Complaint Assessment Team. All NYPD commands are audited twice a year by 
the Complaint Assessment Team. The Complaint Assessment Team sends a team of auditors to 
each audited command to gather the complaint reports selected for audit. Every audit includes a 
review of 25 randomly selected complaint reports in each of 18 classifications of crimes. 
According to the NYPD, the sample size was determined solely on the basis of how many 
complaint reports the Complaint Assessment Team was reasonably capable of auditing for each 
precinct — currently, a maximum of 450 reports. Typically, however, there are fewer than 25 



The categories of complaint reports are: Assault (Misd.); Criminal Impersonation; Criminal 
Mischief; Criminal Possession of Stolen Property; Criminal Trespass; Forgery; Grand Larceny 
(Person); Grand Larceny (Property, including autos); Identity Theft; Lost Property (Unclassified); 
Lost Property (Credit Card); Petit Larceny (A); Petit Larceny (B); Sexual Abuse 1; Sexual Abuse 
2 & 3: Sex Misconduct; Forcible Touching; Investigate Aided. QAD does not audit complaint 
reports in the categories of murder, rape, robbery, and felony assault due to the low risk of 
manipulation in those categories. 

Petit Larceny (A) is petit larceny other than those from a residence or an automobile (e.g., a 
business premises). Petit Larceny (B) is petit larceny from a residence or automobile. 
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total complaint reports in certain categories during the six-month audit period, in which case all 
of the complaint reports for the period in these categories are audited. 

The audit sample is drawn from each month in the six-month audit period; for a 
given crime, four complaints are selected from each of the first five months of the audit period 
and five complaints are selected from the sixth month. The Crime Analysis Unit at a precinct is 
given the list of complaint numbers randomly drawn for review by QAD and has two days to 
provide copies of the files (the scratch reports, any DD-5s, and printed versions of the Omniform 
reports) to QAD. Precincts are not allowed to request substitutions of certain complaint reports 
for others selected for audit. 53 

The QAD audit process typically takes 1 to 3 weeks. The object of the review is 
to ensure proper recording and classification of complaints. The guidelines followed by the 
Complaint Assessment Team are not documented in an audit manual. According to QAD staff, 
the team employs the same general procedures in each audit. 54 Those procedures are as follows: 
(1) each Omniform complaint report is compared to its corresponding scratch copy; (2) any 
discrepancies between the two reports are noted; and (3) auditors note any scratched-out text in 
the scratch copy (for example, changes in the value of lost/stolen property). The audit teams 
review the complaint reports at the precinct, a process lasting approximately three hours. While 
the Omniform versions of all audited complaint reports are taken by the audit teams to QAD's 
offices, the teams copy the scratch versions of only those reports that, in the individual auditor's 
view, require further review. 55 

To test the accuracy of the recorded details in the complaint reports, the team uses 
random or "method" callbacks and non-random callbacks of the complainants. In method 
callbacks, the complainant in every fifth report is called and asked to describe details of the 
reported crime, which are then compared against the complaint report's description. Non- 
random callbacks result when complaint reports contain inconsistencies or have other factors 
(such as a lack of detail) that cause the reviewing officer to question the accuracy of the 
complaint report. The reviewing officer calls the complainant in order to test the accuracy of the 
complaint report. 



Tampering with complaint reports in anticipation of a QAD audit would be apparent to the audit 
team because DD-5s are tracked by Omniform and therefore visible to the review team. 

These statements were supported by the audit documents the Committee reviewed, which were 
uniform (accounting for variations in crime reports among precincts) and appeared consistent 
across the various audits with respect to the errors flagged, certain patterns in errors, and post- 
audit procedures. 

QAD cannot conduct on-the-spot follow-up interviews with police officers due to union 
restrictions entitling officers to representation during interviews. 
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Based on his or her review, the auditor decides whether the complaint report is 
misclassified. There are three types of misclassifications: (i) "as written," meaning that the 
report on its face reflects a crime other than what was classified; (ii) misclassifications 
discovered during non-random callbacks; and (iii) misclassifications discovered during random 
callbacks. Typically, random callbacks uncover few misclassified reports, with as-written and 
non-random callbacks accounting for the majority of misclassification findings. 

In addition to identifying misclassified reports, QAD conducts administrative 
reviews, which check for proper preservation of scratch reports and preparation of Lost/Stolen 
Property and Identity Theft forms. 

At the completion of the Complaint Assessment Team audit, all misclassified 
reports are provided to the QAD lieutenant for review. The lieutenant reviews and initials the 
reports if he or she concurs with the designated misclassifications. Misclassified reports also are 
reviewed and initialed by the Executive Officer of QAD (an inspector), and ultimately reviewed 
by the Commanding Officer of QAD (a chief). 

After the review of the misclassified complaints is completed, they are returned to 
the audit team and a final audit report, which includes the misclassification rate, is prepared. The 
misclassification rate is reported in the final audit report (as well as summary and derivative 
reports) as a rate of the category into which a crime was misclassified. For example, a grand 
larceny misclassified as a petit larceny is reported as an error of the petit-larceny category. This 
method of calculating error rates produces the 1.5% overall error rate quoted by the NYPD. 
Summary reports of all commands produced each audit cycle provide the raw number of reports 
reclassified by their proper classification. For example, in the first audit cycle of 2010, it was 
reported that 71 reclassifications from identity theft would have been grand larcenies. As 
discussed below, this method of reporting errors could be improved. 

The audit report is reviewed by the Deputy Commissioner, Strategic Initiatives, 
who endorses the report. The Deputy Commissioner's endorsements reflect a detailed review of 
the audit findings, including — when warranted — references to specific complaint reports that 
appear to reflect "egregious" discrepancies between the complaint report's description of events 
and what the complainant later told QAD. The Deputy Commissioner's endorsements stress the 
importance of accurate crime reporting and recordkeeping. They also instruct precinct 
commanders to hold officers accountable for failing to comply with crime-reporting regulations, 
including the proper administrative procedures. After the Deputy Commissioner endorses an 
audit report, it is distributed to the Chief of the Department, the Chief of Patrol (Housing or 
Transit), the Borough Chief, and the commanding officer of the audited command. 

Commanding officers are required to respond to the endorsed audit report, and 
they must have Complaint Follow-Up Reports (DD-5s) prepared to correct misclassifications. 
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Copies of the DD-5s must be forwarded to QAD. The Commanding Officer's response is 
forwarded back through the chain of command up to the Deputy Commissioner. 

When both audit cycles are complete, the Complaint Assessment Team re- 
evaluates any command that has an average annual misclassification rate of 2.5% or higher. The 
follow-up includes a three-month review of all complaints in the crime categories with such high 
error rates, using the same procedures followed in the semi-annual audits. In addition, personnel 
at deficient commands, including the officers who filed the misclassified reports, are required to 
attend re-instruction training on crime classification conducted by QAD. 

Confidential Investigations Team. The Confidential Investigations Team, 
comprised of approximately five sergeants supervised by a lieutenant, investigates allegations of 
improper crime reporting. Typically, these investigations begin as the result of complaints filed 
with the Internal Affairs Bureau ("IAB"). IAB assigns an IAB log number to the complaint and 
generates a synopsis of the complaint, which describes the allegations and identifies the subject 
officer. In cases where the allegations involve improper crime reporting, the complaint is 
referred to QAD and the log number and synopsis are forwarded by IAB. The Confidential 
Investigations Team may also initiate investigations in the case of egregious misclassifications 
identified in the course of the Complaint Assessment Team's audits. 

Each Confidential Investigations Team case is unique, and the manner of 
investigation is adapted to address the specific allegations of the complaint. If the complainant 
identifies himself or herself, the team may start by interviewing the complainant. Investigations 
may include a crime-classification assessment similar to that performed by the Complaint 
Assessment Team, but covering a time period and crime categories tailored to the allegations in 
the complaint. Investigations also include formal interviews with members of the command in 
question, including officers who prepare reports, supervisors who sign-off on them, supervisors 
in the Crime Analysis Unit, and civilian employees. Interviews are formal, and officers are 
entitled to bring a representative. 

The Confidential Investigations Team conducts as many as 20 investigations per 
year. Between 2008 and 2012, IAB referred 46 cases to the Confidential Investigations Team, 
and 12 of these resulted in requests for disciplinary action against 42 members of the NYPD. 
Thirty of these members received disciplinary action, and nine are pending disciplinary action. 56 
At the conclusion of their investigations, the Confidential Investigations Team writes a report to 
the Chief of IAB including its investigative findings as to whether the allegations are 
substantiated. The report also describes any misconduct noted by investigators. If the 
investigation included a crime-classification assessment, the Deputy Commissioner endorses the 



In comparison, between 2002 and 2007, there were only 12 disciplinary actions arising from IAB 
referrals. 
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report and distributes it through the chain of command. The Commanding Officer of the 
command in question must respond to the report in the same way required for a semi-annual 
audit. 

SPRINT Audits. In 2010, QAD instituted a new program aimed at identifying 
suppression — SPRINT audits. SPRINT audits are conducted by the Confidential Investigations 
Team. As mentioned above, during a 91 1 call (also known as a "radio run"), certain information 
is tracked and placed in "SPRINT reports," including date and time, job number, and disposition 
of the call. The SPRINT audits test the dispositions of the robbery-related radio runs as reflected 
in the SPRINT reports, including those that are reported by the responding officers as 
"unfounded." 

The Confidential Investigations Team performing a SPRINT audit for a precinct 
pulls SPRINT reports from all 91 1 calls coded as a robbery by the dispatcher in a given week for 
the audited precinct. For those radio runs that resulted in complaint reports, the audit team 
attempts to manually match the SPRINT reports with the Omniform complaint reports and pulls 
the complaint reports. QAD attempts to contact the complainant on each SPRINT report to 
verify the accuracy of the information in the complaint report. Occasionally, QAD calls the 
responding officers to gather more information. 

For SPRINT reports reflecting that the radio run did not result in a complaint 
report (e.g., the call was unfounded, no complainant was present, etc.), the audit team attempts to 
confirm that complaint reports were not required by, among other things, contacting the original 
complainant by calling back the phone number used to report the crime. 

In instances where the Confidential Investigations Team determines that a 
complaint report should have been filed but was not, the precinct commanding officers are 
required to investigate the circumstances of each radio run and either prepare a complaint report 
or explain why no complaint report is warranted. In addition, for any misclassified report 
identified in the SPRINT audit, a DD-5 must be prepared and entered into Omniform. Prior to 
the implementation of the SPRINT audits in 2010, there was only a limited mechanism to 
capture statistics relating to instances in which a complaint report was not prepared in connection 
with radio runs. To date, each command has undergone two SPRINT audits. Beginning with 
the 2013 audit cycles, QAD has expanded the SPRINT audits, previously limited to radio runs 
for robbery, to include radio runs for burglary. As discussed below, the SPRINT audits have 
proven to be a highly effective tool in improving crime-reporting. In addition, because these 
audits involve recorded conversations with complainants — direct evidence of the information 
provided to the NYPD — they are a useful tool in identifying misconduct. 



See infra note 96. 
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Data Integrity Unit 

The Data Integrity Unit, or DIU, is a team charged with reviewing complaint- 
reporting data for inconsistencies. The unit has a staff of 21, which includes a captain, three 
lieutenants, 3 sergeants, 9 officers, and 5 civilians. DIU's work is governed by a written guide, 
and its focus is to look for errors in classifications. Unlike QAD, DIU does not limit the 
complaints it audits to certain crime classifications. DIU reviews for all misclassifications 
(including lateral misclassifications between crimes of equivalent seriousness), whereas QAD's 
focus is upon errors affecting the grades of classifications (i.e., upgrading and downgrading). 

DIU reviews approximately 32,000 complaint reports department-wide per year. 
The entire set of audited complaint reports consists of focused samples and a random sample. 
The focused samples are: 10% of assault-type complaints, e.g., harassment, aggravated 
harassment, menacing, reckless endangerment (which accounts for approximately 9,000 audited 
complaints); all 1st, 2nd, and 3rd degree Rape complaints (approximately 1600 complaints); and 

CO 

all 1st degree Criminal Impersonation complaints (approximately 250 complaints). The 
random sample is comprised of 3% to 5% of all offenses not included in the focused audits. 59 In 
addition to the random/focus complaint audit, DIU reviews all Investigate/ Aided reports, 60 of 
which there are approximately 500 per year. 

In addition to its audit of complaint reports, DIU reviews all non-system- 
generated change reports, or DD-5s, which total approximately 70,000 per year. DIU reviews 
the change reports for the rationale supporting the change. After a change report is finalized at 
the precinct, DIU must approve it. DIU therefore reviews DD-5s within two or three days of 
their creation. When DIU does not approve a change report, it flags the DD-5 for follow-up by 
the amending officer. The only way to remove the flag and complete the change is to 
satisfactorily complete the follow-up request. For example, a DD-5 attempting to change a 
complaint report initially classified as a robbery to a misdemeanor assault might be flagged to 
check the support for the change (such as whether a further interview with the complainant 
revealed that no property was stolen). 

In the DIU audit reports reviewed by the Committee, the upgrade error rate (i.e., the rate of 
misclassifications requiring a corrective upgrade) with respect to Rape 2 nd and 3 rd was 
substantially higher than the error rates with respect to the other focused-sample audits (roughly 
9%-16% compared with approximately 4% rates for assault-related and criminal-impersonation 
complaints). 

Of the random samples audited, the upgrade error rate averaged 3.7%, and upgrades to grand 
larceny constituted roughly 45% of the errors. The large proportion of upgrades to grand larceny 
may be attributed to the large proportion of crimes (e.g., petit larceny, fraud, and lost-property) 
that can be upgraded to grand larceny compared with other crimes. 

An Aided Report records an instance where an officer assists an individual who is injured (not 
involving a motor vehicle), becomes sick, or is lost. 
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The DIU audits occur entirely within a software application that allows the team 
to evaluate complaints and DD-5s. There is no investigation of materials external to the 
information in Omniform (e.g., the scratch reports) as there is in QAD audits. 

Assessment of the CompStat Audit Design 

There are a number of different sources setting forth standards with which internal 
audits should comply. One such source is The International Standards for the Professional 
Practice of Internal Auditing ("Professional Standards") (formerly known as The Standards for 
the Practice of Internal Auditing). To the extent that such standards are relevant, the Committee 
assessed the CompStat audit process in light of the Professional Standards. 61 

Summary of Professional Standards: Auditor Requirements 

The Professional Standards state that the purpose of an internal audit team is to 
provide "independent, objective assurance and consulting services designed to add value and 
improve an organization's operations." The Professional Standards establish requirements for 
the qualifications of the auditors, how the audit should be planned and executed, and how the 
results should be reported. 

(a) Qualification of Auditors 

The Professional Standards state that there should be a "Chief Executive 
Auditor" — which, in the case of QAD and DIU, would be the Chiefs in charge of the units. In 

The Committee selected the Professional Standards to use as a guidepost because The Standards 
for Internal Control in New York State Government, issued by the State Comptroller (available at 
http://osc.state.ny.us/agencies/ictf/docs/intcontrol_stds.pdf), mandate that internal government 
audit functions are "organized and operated in accordance with professional audit standards; in 
particular" the Professional Standards. 

To be sure, the Committee did not undertake to compare the NYPD's processes against the audit 
process of other police departments, however, Smith and Purtell undertook such a survey, supra 
note 20 at 23-24, noting in 2006 that no department responding to their survey had as robust an 
audit program as the NYPD. Nevertheless, the Committee evaluated the NYPD's audit program 
through the prism of the Professional Standards, which reflect the best practices of professional 
auditors in an effort to identify a reasonably appropriate gauge of the department's audit 
procedures. 

The rigors of meeting the Professional Standards may be challenging, and the Committee is not 
aware of any other police departments that utilize them. Nevertheless, and even though the 
NYPD's processes may be more rigorous than other departments, the Committee recommends 
that the NYPD aspire to meet those standards. 

Professional Standards Glossary. 
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addition to possessing the requisite professional background, the Chief Audit Executive must run 
a quality- assurance program and ensure that the audit is meeting performance standards. There 
are two components to a quality-assurance program: (1) an internal assessment and (2) an 
external assessment. The internal assessment includes ongoing monitoring of the audit 
performance. The external assessment is conducted by qualified reviewers from outside the 
organization and should be performed, at a minimum, every five years. 64 

In addition, the Chief Audit Executive must effectively manage the audit to make 
sure that it adds value to the organization. Effectively managing an audit means that the results 
of the audit reflect the purpose and goals outlined in the audit charter, the audit is conducted in 
conformance with the Professional Standards and individuals who work on the audit are in 
conformance with the Professional Standards and Code of Ethics. 66 In order to effectively 
manage the audit, the Chief Executive must (1) establish policies and procedures to guide the 
audit, including risk-based plans; (2) ensure the audit team has appropriate resources to execute 
and perform the audit; and (3) maintain communications with the senior management and the 
board regarding the status of the audit and resources. 67 In order to "add value to an 
organization," an audit must provide objective assurance and it also must contribute to the 
effectiveness and efficiency of governance, risk management and control processes. 68 

There are three main principles that apply to auditors working on internal-audit 
activity: first, auditors must be independent and conflict-free; second, auditors must exercise 
proficiency and professional care; and third, auditors must be qualified and participate in 
continuing professional development. 

(i) Independence 

In order to achieve independence, an auditor, as well as the entire auditing 
function, must be free from conditions that compromise the ability of the auditor to perform his 
or her internal audit activity responsibilities in an unbiased manner. 69 In order to remain 
unbiased, an auditor must maintain — both in fact and in appearance — independence and remain 
conflict-free. An auditor must carry out his or her audit responsibilities with conviction in the 



Professional Standard 1311. 

Professional Standard 1312. 

Professional Standard 2000. 

Interpretation of Professional Standard 2000. 

Professional Standards 2010, 2020, 2030, 2040 and 2060. 

Interpretation of Professional Standard 2000. 

Professional Standards Glossary. 
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work product and cannot make quality or judgment compromises. A conflict of interest arises 
when an auditor has a competing professional or personal interest that impacts, or even appears 
to impact, the individual's ability to fulfill her duties objectively and impartially. 71 

(ii) Knowledge, Skill, and Due Professional Care 

72 

Auditors must perform their work with proficiency and due professional care. 
Proficiency entails possessing the requisite knowledge, skill and competencies to effectively 
carry out one's professional responsibility. 73 For example, auditors must have sufficient 
knowledge to evaluate the risk of fraud. 74 

Exercising due professional care requires auditors to use the care and skill 

75 

expected of a reasonably prudent and competent auditor. The standard of due professional care 
is met when an auditor considers such factors as the extent of work needed to achieve objectives, 
the complexity and materiality of matters to which assurance procedures are applied, the risks 
that might affect the audit, the effectiveness of the governance, risk management and control 
processes, the probability of error and fraud and the cost of assurance in relation to potential 
benefits. In conducting the audit, auditors should consider data-analysis techniques and the use 

77 

of technology. 



Summary of Professional Standards: Audit Planning and Execution 
(i) Audit Charter 

Internal-audit activity ought to be formally defined in a "charter," a formal 

7R 

document that defines the audit's purpose, authority and responsibility. The charter 
"establishes the internal audit activity's position within the organization; authorizes access to 
records, personnel, and physical properties relevant to the performance of engagements, and 



Professional Standard 1100 and Interpretation of Professional Standard 1100. 

Interpretation of Professional Standard 1120, Professional Standards Glossary. 

Professional Standard 1200. 

Interpretation of Professional Standard 1210. 

Professional Standard 1210.A2. 

Professional Standard 1220. 

Professional Standard 1220. Al, A. 3. 

Professional Standard 1220.A2. Auditors must continuously develop their knowledge, skills and 
competencies through professional development. Professional Standard 1230. 

Professional Standard 1000. 
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defines the scope of internal audit activities." The charter should be reviewed periodically and 
approved by senior management. 

In creating the audit charter, the nature of the audit must be considered to ensure 
that the audit activity evaluates and contributes to the improvement of governance and risk 

OA 

management in a systematic and disciplined manner. In terms of governance, the auditors must 
assess and make recommendations for improving governance through promoting ethics and 
values throughout the organization, ensuring effective organizational performance, 
communicating risk information to appropriate parties and ensuring communication among the 
auditors, board and management. 81 For risk management, the auditors must evaluate the 
effectiveness of and improve the risk management process, including the potential for the 
occurrence of fraud and how the organization manages fraud risk. 

(ii) Planning, Executing, and Reporting Audit Results 

The Professional Standards outline requirements that guide the planning and 
relevant considerations for executing and reporting results of the audit. These requirements can 
be broken into six main categories. First, in planning the audit, the objectives of the activity 
being reviewed, as well as the controls inherent in the activity, must be considered. Further, the 
risks, objectives, resources, and operations of the activity should be evaluated, as well as how the 

83 

activity's risk management and control processes could be improved. Second, once the 
objectives have been identified, one must ensure that the scope of the audit is sufficient to satisfy 

84 

the specified objectives. In determining whether the scope is sufficient, one must consider the 
relevant systems, records, and personnel that are associated with the activity being audited. 85 
Third, auditors must ensure that there are appropriate resources to achieve the objectives of the 
audit. Fourth, auditors must develop — and document — an audit work program that achieves 
the audit objectives. The work program includes "procedures for identifying, analyzing, 

oo 

evaluating, and documenting information" during the audit. Fifth, the auditors must base their 



Professional Standards Glossary. 

80 Professional Standard 2 100. 

81 Professional Standard 2110. 

82 Professional Standards 2 1 20 and 2 1 20. A2. 

83 Professional Standard 2210. 

84 Professional Standard 2220. 
Professional Standard 2220. Al. 
Professional Standard 2230. 
Professional Standard 2240. 
Professional Standard 2240.A1. 
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conclusions on appropriate analyses and evaluations. Sixth, audits must be properly supervised 
to ensure that objectives are reached, quality is achieved, and the staff is developed. 90 

The Chief Audit Executive must accurately, objectively, and clearly communicate 
the results of the audit in a complete and timely manner. 91 The communication must include the 
audit's objectives, scope, conclusions, and related recommendations and action plans. 92 



Application of the Professional Standards to the CompStat Audit Processes 

The Committee concluded that the primary, but not the only, risks to the crime- 
reporting process were human error and manipulation (both through downgrading and 
suppression). As described below, while the QAD and DIU audits generally address these risks, 
when reviewed in light of the Professional Standards, there are weaknesses of varying 
significance in the audit processes. 



Strengths of the CompStat Audit Process 

Applying the Professional Standards to the CompStat audit processes, the 
Committee found a number of significant strengths. 

Professional Qualifications of Audit Staff. The professional qualifications of 
the QAD and DIU staff are a strength of the audit program. The audit staff nearly uniformly has 
greater than five years of service in the NYPD. QAD, in particular, consists predominantly of 
supervisory ranks (i.e., sergeants, lieutenants, and captains); and the Complaint Assessment 
Team and Confidential Investigations Team each consists entirely of sergeants and lieutenants. 
Between QAD and DIU, most staff members have at least an associate's degree. There are also 



Professional Standard 2320. 
Professional Standard 2340. 
Professional Standards 2400 and 2420. 
Professional Standard 2410. 

Smith and Purtell, who analyzed the audit program in 2006, though not against the Professional 
Standards, found that it was satisfactory. See supra note 20. The Committee's conclusions differ 
in some respects from Smith's and Purtell' s analysis of the audit program. 

Smith and Purtell compared the NYPD favorably to other police departments. Smith and 
Purtell' s analysis did not compare the NYPD's audit program on a point-by-point basis with the 
Professional Standards, while the Committee attempted such a detailed analysis, even though it 
appears to be a standard to which no other police department has been held. In addition, Smith 
and Purtell did not have the benefit of some of the personnel and materials available to the 
Committee, such as the audit files. 
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seven master's degrees and one Ph.D. in the units. Overall, the experience and educational level 
of the units' staff is more than sufficient to complete the missions of QAD and DIU. 94 

Independence. As described above, auditors must be independent and free of 
conflicts of interest. The Committee identified two primary areas within which QAD and DIU 
operate which may create a conflict of interest: (1) the commands and precincts that those groups 
audit and (2) the Commissioner's office and higher ranks which make use of the audit results. 
QAD and DIU are sufficiently independent from the precincts and officers that they are auditing. 
The officers in QAD and DIU are unconnected with the precincts that they audit, and the 
Committee did not perceive any conflicts of interest inherent in the audit system. QAD and DIU 
report through a chain of command independent of the Patrol chain of command up until it 
reaches the Commissioner's office and the Deputy Commissioner, Strategic Initiatives. This 
structure provides sufficient independence for the QAD and DIU auditors to perform their 
internal- audit functions, with respect to the command and precinct-level. 

With respect to independence from the Commissioner's office, as noted elsewhere 
in this report, the Commissioner and senior NYPD officials face accountability pressures that 
may be similar to what precinct commanders face with respect to reported crime rates. The 
Committee recognizes that QAD and DIU's independence at the level of the Commissioner's 
office is not complete insofar as it may exert influence over personnel, procedure, and practices 
within those units. Nevertheless, as a practical matter, QAD and DIU, as internal audit 
functions, must report into the NYPD hierarchy, and there is no internal authority free of 
potential conflicts (e.g., similar to the audit committee of a public corporation) that would 
completely address such independence concerns. With these reservations, the current reporting 
structure of DIU and QAD with respect to the Commissioner's office, while perhaps not ideal, is 
sufficient. 

Sample sizes. As noted by Smith and Purtell, see supra note 20, audit standards 
generally only require samples large enough to identify systemic errors; the NYPD's sampling in 
its audits exceeds those minimum requirements. As discussed, QAD reviews up to 25 complaint 
reports in 18 crime classifications in each precinct in each audit period, for a total of 
approximately 50,000 complaint reports across all commands of the over 400,000 complaint 
reports filed each year. DIU reviews as many as 32,000 complaint reports per year as well as 
each and every change report (DD-5s). These sample sizes provide tremendous amounts of data. 
This data is useful for identifying common errors, areas for increased training, and other 
potential process improvements. In addition, the review of all change reports prevents gross 
manipulation of complaint reports by blatant misclassifications. 



Despite the professional qualification of the QAD and DIU staff, they do not receive formal 
training in the field of auditing. 
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Continuing Innovation. The NYPD adjusts its audits based on deficiencies it 
discovers in the crime-reporting or the audit processes. QAD recently implemented SPRINT 
audits and created or revised documentation related to crime reporting and auditing. In addition, 
DIU is able to adjust its audit schedules based upon patterns of discrepancies or other errors it 
detects in the reporting process. These innovations reflect positively on the NYPD's continuing 
evaluation of its crime-reporting procedures and safeguards, as well as its good-faith efforts to 
improve the auditing processes. 

Follow-up Investigations. QAD initiates follow-up investigations based on its 
audit findings, most often in cases of "egregiously" misclassified complaint reports. In other 
instances, it initiates investigations upon referrals from IAB or based upon press reports. The 
Confidential Investigations Team, as described earlier, conducts these investigations and pursues 
other leads uncovered during the course of the investigation, expanding as necessary the scope of 
the investigation. The investigative reports describing these investigations are thorough and fair. 
Notwithstanding other issues and recommendations with respect to these investigations noted 
later, as a general matter, these investigations evidence the NYPD's efforts to uncover and 
prevent manipulation in the crime-reporting process. 

Weaknesses in the Audit Design 

Audit Charter. QAD lacks a written audit charter. 95 While QAD' s mission 
appears to be self-evident (e.g., to test and ensure the accuracy of the NYPD's crime reporting), 
the lack of a charter explicitly stating its mission hinders assessments of whether it is achieving 
that mission. For example, QAD's mission could be to uncover all misclassifications, all errors 
affecting crime reporting, or patterns of misclassifications indicating unethical behavior, each of 
which would result in differing performance indicators. Because the NYPD appears to treat 
QAD's mandate broadly, the Committee has assessed QAD's audits as directed at the broadest 
possible mandate — to ensure that the NYPD's crime reporting is accurate for both public crime- 
reporting purposes and for internal crime-management purposes. 

Risk Assessment. As noted above, the Professional Standards require that audit 
plans be based upon assessments of the audited activity's objectives, risks, and existing controls. 
Based on the Committee's discussions with QAD staff, QAD audits were designed without a 
formal risk assessment to identify risks to the reporting process. This is not to say that the 
designers of the audit process were unmindful of the risks — indeed, the audit program is 
designed to capture data sufficient to uncover systemic misclassifications in 18 categories of 
crime. But the lack of formal risk assessment may play a role in several of the deficiencies the 

DIU's "welcome aboard" manual provides a general overview of DIU's mission, which is 
sufficient to serve the purpose of the audit charter under the Professional Standards. Therefore, 
this finding is limited to QAD's processes. 
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Committee found in the audit process. For example, while the risk of suppression is clear, it was 
effectively unaddressed by the audit process prior to the implementation of the SPRINT audits in 
20 10. 96 Further, although the SPRINT audit program addresses the risk of suppression, it does 
not appear that the program was designed following a formal risk assessment. 

Similarly, sample sizes were determined to ensure the accuracy of the seven 
major felonies. But the risk of upgrading crime is relatively minimal compared with the risk of 
downgrading. First, the bias built into the classification rules requiring the most serious 
classification means crimes should tend to be reported in more serious categories. Second, there 
is little or no incentive on the part of officers to upgrade crime. Therefore, as discussed below, 
the lower categories of crime into which most crimes are misclassified, i.e., lost property and 
petit larceny, constitute a large proportion of the audit sample. 

Finally, as noted above, the NYPD continually refines and modifies its audit 
procedures. Many of these changes are reactive to investigations (e.g., the implementation of the 
SPRINT audits) or public criticism (e.g., new reporting guidelines for sex crimes). Despite the 
laudable nature of the refinements to its policies with respect to the SPRINT audits and sex- 
crime reporting, the lack of a formal risk-assessment process and documentation increases the 
risk that the NYPD may not react proportionately to other perceived issues, despite the amount 
of consideration that is apparent in the design and implementation of the new procedures. 

Written Policies and Procedures. As noted above, the Professional Standards 
require written audit policies and procedures. During the Committee's initial informational 
interviews in March 201 1, QAD informed the Committee that it did not have a written protocol 
describing its audit process and procedures, but provided materials used in training newly 
promoted officers. The NYPD subsequently provided a written protocol for QAD audits 
(including precinct SPRINT audits and Confidential Investigations Team investigations) dated 
October 201 1. The QAD audit protocol describes the QAD audit process in step-by-step detail, 
at least on a mechanical basis. The audit protocol lacks, as did the training materials, specific 
guidance on the standard of review employed in evaluating complaint reports. 



There was a precinct-level self-audit procedure in place prior to the SPRINT audits conducted by 
QAD. In the precinct-level self-audit, the Integrity Control Officer compared SPRINT reports 
against the resulting complaint reports. But this review excluded radio runs that did not result in 
complaint reports (i.e., unfounded). The NYPD has changed this procedure such that all radio- 
run dispositions are audited. 

For example, in 2010, there were approximately 81,000 petit-larceny complaint reports and 
75,000 lost-property reports. These two categories comprised approximately 47% of all 
complaint reports and 33% of audited reports. 

See, e.g., John Eligon, Committee Seeks More Police Training on Sex Crimes, N.Y. TIMES, June 
2, 2010 (describing complaints about police officers ignoring or minimizing sexual-crime 
reports). 



36 



DIU provided the user guide for its audit application and related documents, but 
these documents do not describe the process and procedures for determining what to audit or the 
nature of the review. The DIU "welcome aboard" manual provides a general overview of DIU's 
mission, including an audit schedule. The schedule notes that DIU completes each of three types 
of audits on a quarterly basis: "Random/Focus Complaint" audits, "Investigate Aided" audits, 
and reviews of each "Online Complaint Statistical Change" application (referred to as DD-5s). 
None of the documents, however, describes the standard employed in reviewing complaint 
reports for errors. 

The materials provided indicate that the audit procedures and protocols followed 
by both QAD and DIU could be better documented. In particular, although the Reference Guide 
provides the guidelines used to determine classifications, the lack of clear written standards and 
guidance for auditors on what constitutes an error on a complaint report is notable. For example, 
even something as basic as "the described event does not meet the elements of the reported 
crime" is not stated. In addition, although the written materials reflect the various 
misclassification categories (they are: as written, method callback, additional callback), the 
materials lack detail on which complaint reports should be selected for additional callbacks. 

Audit Work Papers. Some of the audit files reviewed by the Committee lacked 
audit work papers tracking which complaint reports were reviewed, which were the subject of 
method and non-method callbacks, and other comments. Some audit files contained these 
papers. QAD personnel stated that whether such files were maintained would have been 
determined by the practice of the individual teams or team leaders. The NYPD now requires that 
such audit files be maintained. In addition, because Complaint Assessment Teams return to 
QAD offices with Omniform copies of all complaint reports, but scratch copies of only those 
reports designated for follow-up, supervisors cannot review complete sets of complaint reports to 
ensure that the auditor selected the proper universe of complaint reports for callbacks. 

Quality Assurance in the Audit Process. Professional audit standards require 
that the audit program be subject to a quality-assurance program able to assess the efficiency and 
effectiveness of the audit program." Such a quality-assurance program should include both an 
internal assessment and an external assessment. The NYPD's ad hoc internal assessment of its 
audit program has led to improvements and refinements in the design of the program, but the 
internal-assessment process should be formalized and documented. In addition, the Professional 
Standards call for external assessments, at a minimum, every five years, conducted by qualified 
reviewers from outside the organization. 100 Currently, notwithstanding this Committee's review 
and the Comptroller's review in 2000, external reviews are not performed on a regular basis. 



Interpretation of Professional Standard 1300. 
Professional Standard 1312. 
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External review of the NYPD CompStat audit process is indeed important and 
may help ease the tension between, on one side, the NYPD's need for accurate and timely crime 
reporting and, on the other, the pressure to report crime decreases. Public skepticism of reported 
crime statistics is rooted in some measure in that apparent conflict, and external review of the 
audit program can contribute to transparency and build faith in the crime-reporting process. 
Transparency, while challenging given the confidentiality of police reports, can also be 
accomplished in a manner that protects the legitimate interests of the NYPD in maintaining the 
confidentiality of its methods and investigations while providing the public with reasonable 
assurance in the effectiveness of the audit program. 101 

Accountability. As described, audit results are endorsed by the Deputy 
Commissioner, Strategic Initiatives, then through the chain of command to the precinct 
commander who must address the audit report's findings. The design of the audit reporting and 
follow-up provides high visibility, as it should under the Professional Standards, to audit results 
and imposes accountability upon underperforming precincts and commands. Moreover, during 
the Committee's precinct visits, the precinct officers stated that they wanted to perform well in 
QAD audits. The NYPD stated that audit results are one of many factors considered with respect 
to a commanding officer' s promotion or assignment. 

Despite these positive aspects suggesting accountability, overall it appears that 
accountability could be enhanced. For example, audit results are not an explicit factor in 
performance evaluations of precinct commanders. With respect to errors attributable to officers 
in the complaint-reporting chain, even in instances of "egregious" errors, the typical resolution 
merely required additional training for the officer or precinct. QAD itself does not always 
interview officers who commit errors, including "egregious" errors; instead, the task is left to 
precinct-level officers. While officers at all levels in the chain of command share responsibility 
for ensuring compliance with NYPD policies and proper reporting, the lack of follow-up 
removes the deterrent effect that an interview by QAD could provide — particularly in those 
"egregious" instances. As we discuss further herein, through 201 1, there were relatively few 
disciplinary proceedings arising from QAD audits, although the rate of disciplinary proceedings 
is increasing due to the SPRINT audits. Thus, while the design appears to foster accountability, 
in practice, there did not appear to be substantial accountability as a result of QAD audits. 

The following chart is a summary of the NYPD's audit program compared against 
the Professional Standards. 



The NYPD is subject to city and state comptrollers' audits as are other city agencies. Only the 
state comptroller has conducted an audit of CompStat. See supra note 21. Under Directive No. 5 
of the City Comptroller, the NYPD would appear to have the authority to implement regular 
outside auditing at its discretion. 

As noted above, in some instances, QAD initiates an investigation by the Confidential 
Investigations Team, in which case the officers are interviewed and disciplined if warranted. 
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NYPD: Application of the Professional Standards to the CompStat Audit Processes 



Professional Standards 


Committee Findings 


A Qualification of Auditors 




1. Chief Executive Auditor: there should be a "Chief 
Executive Auditor" 


In the case of QAD and DIU, this is the 
Chief in charge of each unit. 


Professional qualifications 


Meets or exceeds standards. 


Effective management 

• (1) establish policies and procedures to guide the audit; 

• (2) ensure the audit team has appropriate resources, and 

• (3) maintain communications with the senior management 


Committee recommends creation of 
additional written procedures to guide the 
audit. Resources and communications 
meet standards. 


Quality-assurance program that ensures the audit is meeting 
performance standards 

• Program must assess the efficiency and effectiveness of the 
audit program. 

o An internal assessment (ongoing monitoring of 

audit performance) and 
o an external assessment (external review every 5 
years) 


Quality assurance program has led to 
improvements and refinements, but 
should be formalized and documented. 

There have been two external reviews 
(Committee' s review and the 
Comptroller's review) since 2000, but 
external review could be more systematic. 


Manage the audit to make sure that it adds value to the 

Ul gctillActllUll. 

• the results of the audit reflect the purpose and goals outlined 
in the audit charter, 

• the audit is conducted in conformance with the Professional 
Standards and 


While the design appears to foster 

aeCUUlllaUllliy , 111 piaUUCC, WC ale UllaUlc 

to say, without supporting data, whether 
in fact there is substantial accountability 
as a result of QAD audits . 


• individuals who work on the audit are in conformance with 
the Professional Standards and Code of Ethics. 




2. Independence 




Auditors must be independent 

• An auditor must be free from conditions that compromise 
the ability of the auditor to perform his or her internal audit 
activity responsibilities in an unbiased manner. 


QAD and DIU are sufficiently 
independent from the precincts and 
officers they audit. 


Auditors must be impartial and unbiased 


Meets or exceeds standards. 


• An auditor must maintain independence and remain conflict- 
free. 




Auditors must avoid conflicts of interest 


Meets or exceeds standards. 


• A conflict arises when an auditor has a competing interest 
that impacts the individual's ability to fulfill her duties 
objectively and impartially. 




3. Knowledge, Skill, and Due Professional Care 




Auditors must be proficient. 


Meets or exceeds standards. 


• Entails possessing the requisite knowledge, skill and 
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competencies to effectively carry out one's professional 
responsibility. 




Auditors must exercise due professional care 

• Requires auditors to use the care and skill expected of a 
reasonably prudent and competent auditor. 


Meets or exceeds standards. 


B. Summary of Professional Standards: Audit Planning and 




Execution 


1. Audit Charter 




Charter 

• Internal-audit activity must be formally defined in a 
"charter," which defines the audit' s purpose, authority and 
responsibility. 

• The charter should be periodically reviewed and approved 
by management. 

• The audit must assess and make recommendations for 
improving governance, effectiveness and improve the risk 
management process. 


DIU has a "welcome aboard" manual that 
serves the purpose of the audit charter. 
QAD lacks a written audit charter. 

The NYPD continuously evaluates its 
crime reporting procedures and 
safeguards and revises audits based on 
deficiencies in crime reporting and audit 
iinumgs, uui ii couiQ improve in mis area. 






Planning the Audit 

• Must consider the objectives, risk, resources and operation 
of the activity being reviewed. 


The Committee recommends 
implementing a formal and documented 
risk assessment process. 


Scope 

• Must be sufficient to satisfy the objectives. 


Sample sizes meet or exceed standards. 
The Committee recommends decreasing 
focus on broad volume -based reviews and 
increasing focus on deeper qualitative 
assessments. 


Resources 

• Must be appropriate resources to achieve objectives. 


Meets standards. 


Audit Program 

• Auditors must document and create an audit work program 
that achieves the audit objectives. 


The Committee recommends 

Hpvflnnmpnt of aHHitional written nr\1ir*ipc 

and procedures concerning the audit work 
plan. 


Analysis and Evaluation 

• The auditors must base their conclusions on appropriate 
analyses and evaluations. 


The Committee recommends additional 
written standards and guidance on review 
analysis and evaluation process as well as 
better recordkeeping related to each audit. 


Supervision 

• Audits must be properly supervised to ensure that objectives 
are reached, quality is achieved and the staff is developed. 


The Committee recommends additional 
documentation of each audit to ensure that 
post-audit review is possible. 


Reporting of Results 

• The Chief Audit Executive must accurately, objectively, and 
clearly communicate the results of the audit in a complete 
and timely manner. 


Meets or exceeds standards. 
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Assessment of Audit Practice 



QAD Audit Practice 

QAD provided work papers to the Committee from its 2009 and 2010 audits of 
precincts selected by the Committee: the 13th, 25th, 52nd, 70th, 83rd, 105th, and 120th 
precincts. There were two audits of each precinct in each year. Each set of papers consisted of 
(i) a cover letter from QAD to the Deputy Commissioner, Strategic Initiatives describing the 
findings of the audits; (ii) the endorsement of the Deputy Commissioner to the relevant precinct 
commander; (iii) copies of the auditors' worksheets; (iv) copies of the misclassified complaint 
reports; and (v) the response of the precinct and borough commanders and other officials to the 
shortfalls noted in the audit, including the DD-5s correcting the misclassifications uncovered in 
the audit. In addition, the Committee reviewed complaint reports that were not found to have 
been misclassified from the first audit cycle of 2010 in (i) lost property (unclassified), (ii) lost 
property (credit card), (iii) petit larceny (A), and (iv) petit larceny (B). 103 

As described earlier, each audit included a review of randomly selected complaint 
reports. Each selected complaint report was compared to its corresponding "scratch" report (a 
draft complaint completed by hand by an officer at the scene or the station house) and 
discrepancies between the two reports were noted. In addition to the substantive review of the 
complaint reports, the audits also reviewed whether certain administrative procedures were being 
followed, such as the proper preparation and retention of Lost/Stolen Property Forms and 
whether the scratch copies of complaint reports were preserved. The rate at which precincts had 
failed to maintain the Lost/Stolen Property Forms often exceeded 25%. There were no reports of 
similar issues with respect to the scratch reports themselves (apart from the Lost/Stolen Property 
Forms). Although the error rate was noted in each of the audit reports, apart from promises of 
more training, there did not appear to be coordinated follow-up on this issue. 

With respect to the substantive review of the complaint reports, complaint reports 
were most often found to be misclassified "as written" or as the result of an "additional callback" 
to the complainant. The "method callbacks" (i.e., callbacks conducted for every fifth complaint 
report) did not result in finding many misclassifications. 

The Committee's review indicated that misclassifications generally fell within 
two categories: (1) misclassifications that appeared to be the result of the errors surrounding the 
relative classifications of identity theft, forgery, and larceny in cases involving stolen credit 

1 Given the volume of complaint reports audited by QAD and the administrative burden on the 

NYPD to produce all audited complaint reports from each precinct for all cycles, review of all 
complaint reports reviewed by the Complaint Assessment Team was not feasible. The 
Committee reviewed these categories because the majority of misclassifications pertained to lost- 
property and petit-larceny classifications. 
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cards, Social Security numbers, and attempted fraudulent obtaining of credit cards; and (2) 
misclassifications that appeared to reflect the downgrading of complaint reports with respect to 
robberies, burglaries, and larcenies. 

With respect to the second category, downgrading, certain patterns were apparent 
in how complaint reports were misclassified. 105 For example, there was a pattern of larcenies 
being downgraded to lost property in instances where complainants did not actually see their 
property being stolen. 106 Many of these complaint reports contained language to the effect that 
the complainants did not feel that they were victims of crime. 

Another pattern (of similar volume) was apparent with respect to robberies being 
downgraded to larcenies. In those instances, complainants typically had a phone, iPod, or purse 
snatched from their hands, occasionally following a struggle and an injury, or were accosted by 
multiple perpetrators. The final complaint reports often failed to accurately reflect the element 
necessary to escalate the crime from a larceny to a robbery, such as the forcible nature of the 
taking or the multiple perpetrators. 107 

Finally, the Committee noted a pattern of burglaries that had been downgraded to 
larcenies by the omission of a complaint of an unlawful entry (an element of burglary). 
Instead, these complaint reports noted that there were no signs of forced entry and that the 
complainant's roommates or acquaintances had access to the unit — even where the complainant 
had indicated an unlawful entry, such as through an unlocked window. In addition, attempted 
burglaries evidenced by a broken lock and an attempted breakdown of a door were classified as 



During background discussions between the NYPD and the Committee, the NYPD noted that, 
due to the overlap between the elements of the crimes of identity theft, forgery, and larceny, and 
confusion surrounding the relative grading of such crimes, the NYPD experienced high error rates 
in the classifications of these types of crimes. Multiple revisions to the crime-classification 
manual (including the major revision in June 201 1) and to the complaint-report form itself were 
made to address this confusion. 

The "patterns" identified by the Committee consisted of multiple similar erroneous reports 
whether from the same command or across commands. 

An example was a complainant who boarded a crowded bus or train and placed her wallet in her 
purse after using her Metrocard to board. Later during the trip, the complainant discovered that 
her wallet was missing. During QAD callbacks, the complainant reported that the officers taking 
the complaint report told the complainant that, unless she saw someone take the wallet, it had to 
be recorded as lost property. 

This type of downgrading was also discussed in the 81 st Precinct Report. 

In New York, a person is guilty of burglary when he knowingly enters or remains unlawfully in a 
building with intent to commit a crime therein. N.Y. Penal Law §§ 140.20, 140.25, 140.30. 



42 



criminal mischief. In most cases, QAD discovered these discrepancies as the result of targeted 
callbacks. 109 



Misclassifications (across all commands) were discovered during 18%— 25.5% of 
non-random callbacks. 110 Random callbacks, however, only resulted in a misclassification 
(across all commands) 1.7%-3.4% of the time. 111 There was no documentation reflecting why 
the QAD auditors selected certain complaint reports for additional callbacks. The Committee 
recognized certain fact patterns in misclassified complaint reports, such as snatchings being 
classified as larcenies, or certain pro forma language used in complaint reports, such as "the 
[complainant] did not feel that she was the victim of a crime." Another indicator noted by the 
Committee included instances where the scratch copy initially reflected a higher classification of 
crime that was scratched out in favor of a lower classification. It is possible, given these 
patterns, that the QAD auditors recognized them as indicators of a potential misclassification on 
the face of the complaint report. 

As set forth above, the endorsements of the Deputy Commissioner, Strategic 
Initiatives reflect a detailed review of the audit findings, including references to specific 
complaint reports that appeared to reflect "egregious" discrepancies between the complaint 
report's description of events and what the complainant later told QAD. The endorsements note 
the importance of accurate crime reporting and recordkeeping. They also instruct precinct 
commanders to hold officers accountable for failing to comply with crime-reporting regulations, 
including the proper administrative procedures. 

Each audit resulted in detailed responses from the precinct commander 
concerning any deficiencies uncovered in the audit. These responses typically attributed errors 
to a lack of adequate knowledge or training on the part of the officer. Occasionally, the errors 
were attributed to a lack of proper investigation on the part of the responding officers. Nearly 
every response from the precinct commanders in these instances promised the implementation of 

! ' In addition to these patterns, there were instances of efforts to intentionally downgrade a crime. 

For example, there was a complaint report in which a desk officer scratched out the item values in 
order to bring the total to below the $1,000 threshold for grand larceny. 

Because the Committee only reviewed complaint reports from seven precincts, the effect of such 
patterns overall on the reported crime levels could not be ascertained. 

! The NYPD provided summaries and its analysis of the QAD and DIU audits. These materials 
included summary charts of statistics concerning misclassifications and "Complaint Assessment 
Team Cycle Books" for 2009-2010 and the first cycle of 201 1. The Cycle Books provided 
detailed breakdowns of crime -report statistics by precinct, borough, and citywide. 

' The relatively high success rate of discovering misclassifications during non-random callbacks 
(compared against random callbacks) indicates that QAD auditors are able to identify certain 
potentially misclassified reports based on qualitative factors. It would be advisable to include 
such criteria in a written protocol. 
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remedial training on proper reporting and procedures. Even in instances identified as 
"egregious," the responses by the precincts were similar. The documents do not reflect whether 
any tracking or follow-up was conducted to ensure that the promised actions actually were 
undertaken or whether past issues were considered in the subsequent routine audits. 112 It does 
not appear that prior audit results factor into later audits, e.g., a precinct with high error rates in 
reported larcenies does not result in QAD placing a greater focus on larceny reporting in 

113 

subsequent audits. This lack of accountability is one potential area for improvement in the 
audit process. 

The Committee's review of complaint reports that were not identified as 
misclassified revealed that the majority of such reports were correctly classified without basis to 
question the classification. In addition, there were instances where it appeared that Crime 
Analysis staff in the precincts had corrected errors in classification shortly after the entry of such 
reports into Omniform. Nevertheless, a portion of the complaint reports appeared to the 
Committee to require further review. Such complaint reports not identified as misclassified by 
QAD included instances of complaint reports apparently misclassified as written, or where the 
Lost/Stolen Property Form indicated that property had been stolen in lost-property complaints, or 
mirrored the patterns described above that indicated possible downgrading. The lack of detail in 
the auditor work papers made it impossible to determine in all instances whether callbacks 
occurred. In instances, however, where such papers were available, the work papers indicated 
that some complaint reports warranting follow-up did not receive any. 114 Finally, there were 
precincts whose complaint reports, reviewed in their totality, demonstrated qualitative trends and 

1 The lack of follow-through also raises the possibility that QAD is not adequately considering the 

possibility of fraud. Auditors must consider the risk of fraud. See, e.g., Professional Standards 
1220.A1 and 2120.A2. The patterns in the misclassifications identified by QAD and observed by 
the Committee, discussed above in this section, raise a possibility of intentional manipulation that 
may warrant further auditing or investigation. 

1 While QAD does focus on problematic categories when conducting focused audits, such audits 

only occur when a command has an average error rate exceeding the error rate for all commands. 
Moreover, even this approach does not appear to factor into the later regular audit cycles. How 
such prior audit results should factor into future audits should be addressed during the audit 
planning phase. See, e.g., Professional Standards 2201. 

In addition, some endorsements do note improvements or declines with respect to compliance 
with administrative procedures, for example, the percentage of improperly prepared/missing 
Lost/Stolen Property Forms. 

One lost-property complaint report, in particular, stood out. The narrative section described a 
complainant who "lost property" following an assault by multiple individuals. On its face, the 
narrative appears to describe a robbery. The work papers in connection with the QAD audit did 
not reflect that this complaint report was flagged for further review. During an interview, QAD 
personnel acknowledged that the complaint report should have resulted in a callback. According 
to these officers, the narrative was insufficiently detailed to support the classification, but further 
facts might support the lost-property classification. 



44 



patterns such as those identified above, which were indicative of the need for follow-up targeted 
audits. 



Disciplinary Action 

Officers are disciplined as a result of QAD findings or IAB referrals based on 
crime reporting. Records provided to the Committee by the NYPD indicate disciplinary action 
or pending action arising out of IAB referrals in 20 cases against 53 members of the service 
between 2002 and 2011. About half of those disciplinary determinations were handed down as a 
result of investigations begun in 2010 and 201 1. All were administrative in nature, although the 
NYPD did confer with District Attorney's Offices in connection with considering criminal 
charges in some instances. In addition, the NYPD disclosed information about seven pending 
cases arising out of QAD-initiated investigations during 2010 and 201 1. 

With respect to IAB referrals, such referrals covered a broad range of precincts 
and ranks. While some were generated by civilians who attempted to report crimes, many 
appeared to be generated by police officers. 115 The Confidential Investigations Team's 
investigations were thorough, impartial, and well documented. The Team expanded 
investigations in light of new leads or potential wrongdoing uncovered in the course of their 
investigations. They utilized investigative tools such as targeted complaint-report audits, 
SPRINT audits, and interviews of involved personnel and complainants. 

Notwithstanding the overall high quality of the Confidential Investigations 
Team's efforts, it appeared in some earlier instances that investigations were tailored too 
narrowly to specific and literal interpretations of allegations. For example, in instances where 
suppression was alleged prior to August 2010, the Confidential Investigations Team reviewed 
SPRINT runs that resulted in a disposition of "complaint report prepared." This approach failed 
to account for the possibility that officers intending to suppress a complaint report may have 
reported a disposition in which a complaint report is not required, e.g., unfounded. The 
Confidential Investigations Team has since changed its approach in such instances. Since this 
change, two of the four investigations of such claims have substantiated the allegations of 
misconduct. In addition, the Confidential Investigations Team relied upon divergence from the 
city-wide error rate as grounds for finding general precinct-wide allegations unfounded or 
unsubstantiated. But in other instances where misconduct was uncovered, those precincts' error 



In one instance, a precinct commander instructed a sergeant to call IAB in response to what 
appeared to be missing scratch copies of complaint reports. After investigation, no wrongdoing 
was uncovered. 
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rates did not diverge significantly from the city-wide rate. In other words, divergence from 
the normal error rate does not appear to be indicative of intentional manipulation. 117 

A substantial number of the cases appeared to be instances in which there were 
facts suggesting manipulation, which ultimately were deemed unsubstantiated due to a lack of 
evidence. In a number of instances, the Confidential Investigations Team located complaint 
reports in Omniform that were alleged to have been suppressed. 118 In other cases, good-faith 
misunderstandings resulted in errors that appeared to be the result of malfeasance. 

As with the QAD audit files, there were notable patterns and trends. For example, 
when interviewing officers during investigations, Confidential Investigations Team members 
asked whether the officers felt any pressure to suppress or downgrade crimes. Nearly uniformly, 
officers stated that they faced no such pressure — even in cases where downgrading and 
suppression were found to have occurred. 119 In investigations uncovering suppression or 
downgrading, there was also a trend that complaint reports were more likely to be suppressed or 
downgraded where it appeared that the complainant could not identify the perpetrator or where 
the complainant did not have time to assist the police. The Committee did not see any 
indications that this trend was addressed in the NYPD's current audit processes. Such trends 
should be considered when conducting future risk assessments and audits. Finally, in a number 
of cases in which a complaint report was not taken, but recorded after investigation by the 
Confidential Investigations Team, the NYPD subsequently arrested suspects in the underlying 
crime. 

Given the difficulty of proving intentional downgrading or suppression, it is not 
surprising that QAD investigations have not had more serious consequences such as referral to 
criminal prosecutors or termination of employment — although some officers have resigned or 
retired rather than face charges. As discussed, crime reporting requires subjective judgments 
limited by the circumstances and abilities of complainants and responding officers. In light of 
these limitations and the ready excuses that they provide, proving an intentional misclassification 
is a significant task. Nonetheless, increased internal investigations might provide a deterrent 



This factor (lack of divergence from the city-wide error rate) also does not take into account 
suppression (which does not factor into the error rate). Nor does it account for the qualitative 
nature of the errors — for example, were they egregious — which could indicate whether further 
investigation is required. 

As discussed further below, a low error rate can mask a larger proportion of downgraded index 
crimes. 

For example, a complainant might report that a police officer failed to take a report, but the 
system shows that a complaint report was taken on the date of the incident. 

In one instance, however, a detective admitted manipulating complaint reports — not because of 
an express instruction from anyone to do so, but because he thought he was supposed to do so. 
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effect, demonstrate the importance of the accuracy of crime reporting, and perhaps uncover 
unethical behavior. 

Summary of Assessment of QAD Audits 

In sum, the Committee's review of the QAD audits raises several issues. First, 
while it is apparent that the current audit process can detect errors and downgrading — in part on 
the basis of patterns in misclassifications — the documentation does not reflect that QAD 
adequately pursues patterns that may indicate downgrading practices by a precinct or particular 
officers. Second, the patterns of the misclassified reports support in some measure the anecdotal 
accounts of downgrading of certain types of incidents, including that certain types of incidents 

120 

may be downgraded as a matter of practice in some precincts. As will be discussed, such 
misclassifications would materially affect the reported statistics. Third, the persistence of 
"egregious" errors in certain precincts despite the pre-finalization review of the complaint reports 
by supervisors may be construed to support the conclusion that complaint reports are not 
meaningfully or at least proficiently reviewed at the precinct level — or, in the worst light, that 

121 

the reviewing supervisors are complicit in the downgrading. Fourth, follow-up investigations 
and disciplinary enforcement can be improved to enhance the accountability that should be 
imposed by the audit program. Fifth, QAD may not flag complaint reports warranting follow-up 
in every instance or identify patterns in complaint reports indicating potential issues, leaving 
errors undetected as a result. Sixth, precinct audits lack the documentation to conduct a proper 
quality-assurance review to detect such auditor errors. 

With respect to these issues, QAD's follow-up audit procedures focus on 
quantitative factors (i.e., the error rate) instead of qualitative factors (i.e., the nature of the 



In their best light, these incidents may reflect questionable job performance by (i) officers 
responding to the relevant complaints, including poor interviewing of victims and inadequate 
record-keeping, (ii) the supervisors who reviewed and signed-off on the complaint reports, and 
(iii) the supervisors responsible for the responding officers' performance. 

During interviews, QAD personnel indicated their opinion that most errors were not the result of 
malfeasance, but rather training or workload-related issues. They provided a notable example of 
an overloaded desk officer. Desk officers are required to sign-off on each complaint report before 
the end of their tour. Omniform allows for desk officers to effectively sign-off on complaint 
reports as a group without review. An overloaded desk officer thus may finalize an entire group 
of complaint reports with little or no review. There is no way to determine how thorough a 
review the desk officer gave each report prior to finalizing it within Omniform. 

As indicated in the Committee's discussions with Eterno and Silverman, as well as in the case of 
the substantiated misconduct in the 81 st Precinct, intentional downgrading likely would be the 
product of supervisory direction. A number of pending disciplinary actions are for desk officers 
and supervisors reportedly failing to review complaint reports or directing officers to file 
improperly classified complaint reports. 
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errors). As illustrated by the circumstances in the 81st Precinct, where the error rate in QAD 
audits was low, a purely quantitative approach may not uncover instances of intentional 
manipulation, particularly suppression. As noted below, the error rate relied upon by the audit 
teams may conceal material differences in crime rates because of the far larger number of 
misdemeanors and lost-property complaints compared with those in felony categories. In other 
words, it is possible to downgrade a small percentage of misdemeanor and lost-property 
complaint reports, yet still significantly affect crime statistics in the seven major felonies. 

The Complaint Assessment Team may refer instances of apparent intentional 
downgrading to the Confidential Investigations Team. Since 2007, such referrals have ranged 
from two to twenty per year, compared with the approximately 800 misclassifications discovered 
during the annual audit cycles. While not all misclassifications give rise to suspicions of 
potential manipulation, there is little risk that any particular instance of downgrading will be 
uncovered. Even if uncovered, given the infrequency of referrals for enforcement actions, there 
is a significant possibility that low-grade manipulation will not result in disciplinary action. The 
Committee recognizes the difficulties in procuring evidence sufficient to establish that a 
particular misclassification was intentional, the finite resources available to QAD, the limitations 
on investigative discretion imposed by these factors, and the good-faith efforts of QAD to 
identify and fully investigate instances of manipulation. Yet the deterrent effect of potential 
disciplinary action may be mitigated by the unlikelihood of discovery and sanction. 

Despite these deficiencies, two efforts on the part of QAD should be noted. First, 
QAD does attempt to address and account for patterns indicating downgrading. For example, 
revisions to the Crime Classification Manual address some of the patterns apparent in the 2009 
and 2010 audits. QAD also has tested and implemented additional procedures and investigative 
techniques, such as SPRINT audits. Second, QAD investigations and enforcement have 
increased in recent years and, according to those NYPD officers the Committee interviewed at 
the precincts and in QAD, enforcement is making a difference in the complaint-reporting culture. 
Officers are taking their obligations with respect to complaint reporting more seriously and with 



In the initial NYPD presentation provided to the Committee, the NYPD noted that QAD 
conducted targeted audits in precincts where the precinct's misclassification rate exceeded the 
average rate in 2010. Such targeted audits focused on the crime classifications that caused the 
misclassification rate to exceed the NYPD average. There were 16 such targeted audits in 2010. 

In addition, in 2010, 12 new cases were referred to the Confidential Investigations Team ("CIT") 
by IAB and six cases were self-initiated by QAD. 181 NYPD members were interviewed by CIT 
in 2010. In 2012, 20 investigations were self-initiated by QAD. These metrics indicate that 
possible instances of downgrading are being pursued with increasing frequency. 

The Committee notes, and credits, the statements by the officers in the precincts that we visited 
that sustained schemes to manipulate complaint reporting inevitably would be discovered and it 
was not worth the risk of undertaking such a scheme. 
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greater awareness that abuses will be addressed internally. Still, for the reasons described 
below, further adjustments are recommended. The Committee recommends that QAD 
incorporate qualitative factors into its determinations for follow-up audits of precincts and 
consider altering the balance of audit resources to allow for more thorough investigations of 
potential manipulation. 

The Applications and Implications of the QAD Audit Statistics 

Another concern of the Committee is the NYPD's internal and external reporting 
of its audit results, particularly in the presentation of its overall error rate as being 1.5%. As 
previously described, the NYPD reports errors as a rate of the crime classification into which the 
crime is misclassified. As discussed further below, this method of reporting provides no 
visibility into the effect of misclassifications upon the properly classified categories. The 
absence of this data from reports of the error rate diminishes the import of the error rate itself in 
evaluating the overall accuracy of the crime-reporting system. Moreover, by not providing equal 
visibility into the impact of the rates of the properly classified categories, efforts to improve the 
complaint-reporting process and the auditing process may be impeded, and the overall 
transparency of the CompStat process may be subject to question. 

A close review of the NYPD's statistics and analysis demonstrate that the 
misclassifications of reports may have an appreciable effect on certain reported crime rates. For 
example, in 2009, the error rate for lost-property complaint reports was 2.2%, of which 95% of 
the misclassified crimes should have been grand larcenies. There were 76,733 lost-property 
complaints in 2009. Subject to the reservations discussed below, applying the error rate across 
all lost-property complaint reports, there would have been 1,603 additional grand larcenies 
reported in 2009. Applying the same methodology to all classifications affecting grand larceny, 
there would have been an estimated 1,006 additional grand larcenies reported in 2009. After 
subtracting misclassified grand larcenies and adding misclassified grand larcenies of autos, there 
would have been 2,312 additional reported grand larcenies or a 4.6% increase (although still 
lower than the unadjusted 56,724 combined grand larcenies reported in 2008). The chart 
below compares the effect of misclassifications across selected felonies, listed in order of their 
respective gradations for purposes of reporting. 



During the Committee's interviews, NYPD personnel indicated that recent press coverage of 
crime-reporting issues had raised awareness and served as a deterrent to misconduct. 

The additional grand larcenies would have been 2 upgrades from criminal impersonation, 34 from 
criminal mischief, 13 from criminal possession of stolen property, 74 from forgery, 252 from 
identity theft, and 63 1 from petit larceny. 

These adjustments are factored into the charts below as reflected by the "(adj.)" in the crime- 
classification column. 
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2009 




Crime classification 


# of Complaint 
Reports 


Estimated Additions 
(applying general 
error rates) 


Error Rate 


Robbery 


18,601 


288 


1.5% 


Felony Assault 


16,773 


619 


3.6% 


Burglary 


19,430 


1,020 


5.2% 


Grand Larceny (adj.) 


50,269 


2,312 


4.6% 




2010 


Crime classification 


# of Complaint 
Reports 


Estimated Additions 
(applying general 
error rates) 


Error Rate 


Robbery 


19,486 


417 


2.1% 


Felony Assault 


16,956 


782 


4.6% 


Burglary 


18,600 


1,033 


5.5% 


Grand Larceny (adj.) 


48,435 


2,625 


5.4% 



In the above charts, the error rate (e.g., robberies were underreported by 2.1% in 
2010) reflects the number of downgraded crimes in each category (e.g., there were 
approximately 417 downgraded robberies in 2010) and approximates the potential percentage 
increase over the reported crime rates in those years. The Committee observed that, generally 
and for whatever reason, the more serious the felony, the less likely it was that the crime was 
misclassified. Analysis of the year-over-year changes in reported crime against the 
extrapolated numbers demonstrates that there would be some effect on the reported annual 
changes in reported crime, but that the trends were generally accurate. Between 2009 and 2010, 
robberies were reported to have increased by 4.8%, felony assault increased by 1.1%, burglary 
decreased by 4.3%, and grand larceny decreased by 3.6%. Using the extrapolated figures, 



Such observations may affect audit planning because the error rates of felony assault, burglary, 
and grand larceny appear to be double that of robbery. 
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robberies would have shown a 5.4% increase, felony assaults a 2% increase, burglary a 4% 
decrease, and grand larceny a 2.9% decrease. 

This analysis, however, is subject to limitations, including those noted in this 
report about crime reporting generally and CompStat specifically, and must be considered in the 
appropriate context. For example, the addition of the extrapolated figures to the reported crimes 
would not create a true or actual crime rate because it would not account for unreported crime, 
good-faith subjectivity in classifications, suppressed complaint reports, or undiscovered errors. 
This observation illustrates that the reported crime rate is an inexact measure of actual crime. 
Nonetheless, we believe that the extrapolations are important because they demonstrate that the 
reported crime rates and audit error rates themselves may be inaccurate and misleading 
barometers of the level of actual crime, whether crime is increasing or decreasing from year to 
year, and of the NYPD's performance. 

First, the NYPD's error rate focuses on the percentage of a category into which a 
crime is misclassified and does not reflect the effect of the misclassification on the properly 
classified category. By failing to give this statistic equal emphasis, full transparency into how 
the crimes are downgraded (whether intentionally or otherwise) is diminished because the 
resulting error rate is a lower number due to the larger number of misdemeanors and lost- 
property complaint reports. For example, the NYPD has focused on the proper classification of 
identity theft vs. forgery vs. larceny because of the "high rate" of misclassifications of identity- 
theft complaint reports. But, in fact, identity theft only represents 1 1% of grand-larceny 
misclassifications. Misclassified grand larcenies are improperly classified as lost property 60%- 
70% of the time. These figures support the misclassification patterns present in the audit files 
provided by the NYPD. Analysis of misclassifications in this manner may suggest further 
avenues for the NYPD to assess misclassification risk and target patterns of misreporting that are 
not otherwise addressed by the current auditing practices. For example, it is clear that 
misclassifications into lost property as a purely quantitative matter have a greater effect upon 
crime statistics than misclassified identity thefts. 

Second, when error rates are calculated in the manner set forth in the charts above 
instead of in the format in which they are customarily presented, a more accurate picture of error 
rates emerges and contributes to a more transparent process. This accuracy and transparency 
should go a long way in addressing what appears to be the public's greatest concerns: (1) by how 
much are major-crime rates affected by classification errors and (2) are the reported decreases in 
such rates credible? 



There still may be qualitative reasons for focusing on identity-theft classifications. Analyses of 
such considerations are prime examples of the reasons why a periodic risk-assessment process is 
necessary. 
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Third, these error rates only reflect downgrading, not suppression. As discussed 
below, the initial results of the SPRINT audits reveal that the risk of suppression (as measured by 
its effect on reported crime) may be more substantial than the risk of downgrading. 



SPRINT Audits 

QAD provided documents from fifty-four SPRINT audits conducted in 2010 and 
201 1. In addition, the NYPD later provided summary results of completed SPRINT audit cycles 
in 2011 and 2012. Following the SPRINT audits, the Deputy Commissioner, Strategic Initiatives 
issued endorsements requiring follow-ups at the precinct level similar to those following QAD 
audits. Responses were also similar in nature to those in the ordinary QAD audits. Overall, the 
SPRINT audit results provided to the Committee reflected that 17% of reportable events went 

i on 

unreported during radio runs for robbery in 201 1, compared with 13.5% in 2012. In addition, 
4.7% of the existing complaint reports reviewed during the SPRINT audits in 201 1 were 
misclassified from robberies to lesser offenses (4 to grand larceny, 3 to assault, and 3 to petit 
larceny), compared to 1.9% in 2012. Similar to the QAD audits of complaint reports, certain fact 
patterns emerged in unreported robberies. Some representative examples of these patterns 
include: 

• Responding police officers interviewed the complainant, then left the 
complainant to "canvass the area for suspects" without taking a complaint 
report (either before or after returning from the canvass). 

• Although the responding officers were required to take a complaint report 
regardless of whether the victim cooperated, officers asked the victim 
whether he or she wished to file a report, and the victim declined. 

• Responding officers took statements leaving complaining witnesses or 
victims with the impression that a report was being filed, but then did not 
file a complaint report. 

Prior to the implementation of the SPRINT audits, no mechanism existed to 
capture these statistics. Given the apparent error rate in radio runs (17% to 13.5% in which 
reports were not filed), this is a higher-risk category for errors than misclassified complaint 



Approximately 56.1% of the radio runs audited in 2011 and 55.7% in 2012 should have resulted 
in some manner of complaint report, based on the combined determinations of responding 
officers and SPRINT auditors. The Committee treated as unreported any case in which the 
responding officer' s disposition indicated a report should be prepared and no corresponding 
report was located, and any case in which the disposition indicated no report was necessary, but 
auditors later determined that a report was warranted. 
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reports, particularly considering that unreported radio runs do not appear in any crime category 
and are not documented other than in SPRINT reports. Since they began in 2010, SPRINT 
audits have resulted in a relatively high rate of disciplinary action, with investigations of 71 
commands in 2011 and 57 commands in 2012, and administrative sanctions sought against 173 
officers. The Committee recognizes that the SPRINT audit program is in its nascent stages and 
that it may take more time before the significance of the information yielded by the audits can be 
fully understood and acted upon. Nevertheless, the SPRINT audits (or their underlying 
methodology) may be one of the best means to identify downgrading and suppression, and as set 
forth in the Recommendation section below, ought to be expanded and refined, if necessary. 



DIU Audits 

DIU audits occur entirely within the computer application used by DIU to audit 
complaint reports. Production of the associated records to the Committee was not practical due 
to the large volume of audited complaint reports and the fact that the audits occur entirely within 
the computer system. Instead, DIU produced copies of the various audit reports. Each audit 
report consisted of a short summary report and tables of error rates by precinct and crime. 
Notably, the 2010 random audits resulted in an error rate of 3.7%, approximately half of which 
required an upgrade to one of the seven major felonies. Assault-related crimes resulted in a 3% 
error rate requiring upgrading to felony assault. As discussed in connection with the QAD 
audits, error rates in the misdemeanor categories would have larger effects on the felony 
categories due to the greater ratio of misdemeanors to felonies (e.g., a 3% error rate in 
misdemeanor assault might result in an 8% increase in reported felony assaults 130 ). 

There was no documentation indicating what, if any, follow-up or actions resulted 
from these audits. 



DIU's focused audit of assault-related crimes in 2010 revealed a 3% error rate. There were 
approximately 45,600 misdemeanor assaults in 2010. If 3% of those were actually felony assaults 
as suggested by the DIU audit, there should have been 1,368 additional felony assaults in 2010 in 
addition to the 17,054 reported by the NYPD. In other words, felony assaults may have been 8% 
higher than what was reported in 2010. This methodology is different from that applied earlier in 
this report, but the implications are similar, i.e., the reported numbers are not exact. 
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A Final Note on the NYPD's Audit Practices and Crime-Reporting 

As we have noted throughout this report, crime-statistics reporting suffers from 
many limitations, risks, and weaknesses. To be sure, CompStat — both the process and the 
statistics underlying it — is an extremely useful tool for law enforcement. But the focus of the 
NYPD and the general public on the year-over- year declines in crime as reported by the NYPD 
can, if over-emphasized, serve to undermine the integrity of the statistics and compromise 

131 

CompStat as an effective law-enforcement tool. This intense public focus is a product of the 
politicization of year-over- year declines in crime that are growing ever smaller (and have now 
"increased")- 132 Although no testing has been done to quantify these factors, the effects of 
unreported crime, the subjectivity inherent in crime classifications, the shifting procedures and 
rules for classifying crimes, and downgrading and suppression necessarily vary in any given 
year. Thus, from the point of view of the public, slight movements in amounts of reported crime 
likely are not sufficiently reliable to be a measure of whether crime is actually rising or falling at 
such small rates or measurably affecting the quality of life. 

From the point of view of the NYPD, however, CompStat is intended to be used 
as an analytical tool in its mission to serve the public, and as such, slight statistical variations 
may be of import to its crime analysts. Thus, understanding the root causes of misclassifications 
and eliminating those that can be controlled should be a critical part of the NYPD's use of 
CompStat. By promoting these statistics as a measure of its effectiveness, any law-enforcement 
agency increases the likelihood of misclassifications and decreases the utility of CompStat as a 
law-enforcement tool. The improvements to the NYPD's internal-audit program recommended 
in this report may mitigate the effect of this focus, but cannot address the cause. 

Finally, we note that, even considered in light of the Committee's findings, the 
accomplishment of the NYPD in reducing crime compared with twenty and even ten years ago is 
indeed historic. 



Concerns such as these are well articulated in FBI publications concerning the UCR statistics. 
See supra note 5. The FBI discourages the use of crime -reporting statistics as a measure of law- 
enforcement effectiveness. 

The total number of reported major crimes increased between 2010 and 201 1 . See 

http://www.nyc.gov/html/nypd/downloads/pdf/analysis_and_planning/seven_major_felony_offen 

ses_2000_2011.pdf. 
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COMMITTEE RECOMMENDATIONS 



Although the detailed recommendations below primarily focus on QAD's 
processes, officers at all levels of the chain of command and at the precinct, borough, and 
headquarters level share responsibility to provide accurate reporting, promote integrity, and 
prevent unethical behavior. The NYPD should consider the extent to which the 
recommendations can be applied throughout the NYPD. 

Documentation. QAD and DIU Audit processes can be improved with better documentation, 
including: 

• A written charter for QAD; 

• A risk assessment to determine audit scope and plans; 

• Increasing detail in protocols by, for example, setting forth the "standard of review" 
employed in evaluating complaint reports; 

• Drafting audit plans to address risks identified in the assessments and prior audits of 
specific precincts; 

• Improving documentation by maintaining work papers and full sets of reviewed 
complaint reports in high-risk classifications; 

• Improving post-audit reports to increase accountability for "egregious" errors. 

Precinct Documentation. Each command should be required to keep a log of requests or 
inquiries concerning complaint reports. The log should reflect the date of the inquiry, the names 
of the inquirer and the complainant, the date of the incident, sufficient detail concerning the 
incident to enable its discovery within Omniform, and the result of the inquiry, including the 
complaint-report number if located. QAD should audit this log during each precinct audit. 

Risk Assessment Processes. The Committee recommends that the NYPD consider the 
following risk-assessment processes. 

• First, the NYPD should consider the creation of a general risk assessment related to the 
crime-reporting process generally. Ideally, this document should be consistently updated 
as issues emerge and should be consulted periodically in connection with the refinement 
of the audit processes. 
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• Second, the NYPD should consider individualized risk assessments for each QAD audit. 
The benefits to such individualized risk assessments are risk-based audits that may prove 
more efficient and effective at ensuring that the CompStat data is reliable. On the other 
hand, it may be that such individualized risk assessments remove the ability to compare 
error rates on a uniform basis or otherwise impact other goals of the NYPD. If a 
precinct-level risk-assessment process is not adopted, the Committee strongly 
recommends the enhancements in the QAD audit process described below. 

o Under the current audit design, which involves twice- yearly audits, such a task is 
likely impractical. The Committee recommends that the NYPD consider 
conducting one annual uniform audit cycle and that the second audit cycle focus 
on specific individualized findings from the standardized audit. 

Enhancing QAD Audit Process. The current document-focused audits conducted by QAD and 
DIU do not sufficiently address the risk of downgrading and suppression — particularly the 
detection of practices and processes at the precinct level that affect the accuracy of complaint 
reports. 

• QAD should add standard officer interviews during precinct audits to address directly 
training or practices that undermine the accuracy of crime reporting. 

• The NYPD should change QAD's procedures to incorporate qualitative factors (e.g., 
repeated errors in certain classifications, certain patterns in misclassifications) into its 
determinations for follow-up audits of precincts instead of or in addition to the purely 
quantitative factor it currently uses (i.e., error rate of the precinct compared with the 
NYPD average). 

• QAD staff should be more proactive in pursuing errors that suggest intentional 
downgrading. 

• In order to enhance the QAD audit process, the NYPD should consider reducing the total 
number of complaint reports audited. Such a reduction may allow for more in-depth 
qualitative risk-based audits and better investigations and follow-up efforts with respect 
to potential manipulation, poor training or supervision, and officer indifference. 

o Any such reduction or restructuring of the audit process should be conducted in 
light of a comprehensive risk-assessment process as outlined above. 

Auditor Training. The Committee recommends that QAD and DIU staff receive audit training 
at least at the supervisor level. Such training will increase sensitivity to issues identified in this 
Report, improving long-term planning in addition to actual audit activities. 
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Accountability. There appears to be a lack of accountability with respect to the findings in 
endorsements, even considering increased enforcement in recent years. Even where errors are 
noted to be "egregious" by the Deputy Commissioner, the resulting action is often promised 
training of the officer. The NYPD should institute a means of tracking audit-by-audit 
performance and incorporating it more formally into the post-audit process in order to hold the 
appropriate personnel responsible for failings (e.g., often it appeared that only the responding 
officer was trained, despite reviews of the "egregious" complaint report by supervisors). In 
addition, the NYPD should consider having QAD interview officers who misclassify crimes to 
evaluate the risk of intentional manipulation. In instances where the QAD decides that such an 
interview is unnecessary, QAD should document the reasons the error is not being investigated 
in its work papers (e.g., the facts were recorded correctly in the complaint report and the error 
was not egregious). 

• In addition, the NYPD should report on a consistent basis to the entire service the errors 
that QAD is discovering and any disciplinary sanctions resulting from such 
misclassifications. Without consistent reporting, the NYPD is missing a prime 
opportunity to educate officers and deter manipulation. 

External Assessment. As the Professional Standards indicate, periodic external assessments are 
a vital part of any internal-audit program. The NYPD should formalize a periodic external 
review of its CompStat audit program. 

Increased Internal Investigations and Enforcement. Increased internal investigations might 
provide a deterrent effect, demonstrate the importance of accurate crime reporting, and 
potentially uncover unethical behavior. Investigations and discipline have increased over prior 
years. The standards for investigative referrals should be documented and should be robust 
enough to ensure appropriate levels of enforcement action. 

• As previously described, QAD staff should interview officers responsible for 
misclassifications to determine the risk of potential fraud. In instances where QAD 
determines that such an interview is not required, QAD should record the basis for that 
determination. 

• Consideration of QAD audit results during investigations should account for the fact that 
a low overall error rate can hide a higher major-crime error rate. The error rate is not 
indicative of qualitative factors within misclassified reports that may suggest 
downgrading (e.g., a pattern of noting that lost property complainants "did not feel that 
they were victims of a crime"). 

• Since the Committee's work began, the NYPD has expanded the use of SPRINT audits in 
investigations, in particular expanding such audits beyond radio-run dispositions of 
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"complaint report prepared" in instances where suppression or downgrading is alleged. 
The NYPD should continue to assess and evaluate further improvements to its 
investigative techniques in these areas. 

SPRINT Audits. Given the results of the SPRINT audits to date (both in terms of the high error 
rate detected and resulting disciplinary actions) as well as the fact that SPRINT audits provide a 
systematic means to detect suppression of crime reports that does not exist in the current QAD 
audit process, the NYPD should devote more resources to the SPRINT audits and expand them 
beyond robbery and, as of 2013, burglary, subject to a risk-assessment process. 

Reporting. In addition to the currently reported error rate, which primarily reflects how much 
lesser crimes are overstated, the NYPD should publicly report approximate error rates by index 
crime. This is a far more transparent practice. In addition, the NYPD's internal reporting should 
expressly reflect such rates. 
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APPENDIX A: Key Terms and Concepts 



This report addresses a number of technical areas and concepts. For the sake of 
clarity, we summarize certain key terms and concepts here that are discussed in this report. 

CompStat. CompStat is system of management that utilizes computerized crime- 
reporting data to assess current crime patterns and trends in varying levels of detail in order to 
determine strategies for combating crime. The system, in addition to making data easily 
available to commanders at all levels through custom and regularly published reports, consists of 
periodic meetings ("CompStat meetings") between precinct commanders (and members of their 
commands) and top NYPD officials (including Deputy Commissioners, various chiefs, and 
Borough commanders). Other government officials, such as representatives from the various 
District Attorney's Offices, the United States Attorney's Offices, and other local and federal law- 
enforcement agencies, might attend CompStat meetings. 

Crime. For purposes of this report, crimes are conduct prohibited by the New 
York Penal Law and classified therein as felonies or misdemeanors. The most severe crimes are 
felonies (e.g., murders), which are defined as such because they are punishable with prison terms 
of more than one year. Misdemeanors (e.g., petit larceny) are crimes punishable by more than 
fifteen days, but less than a year in prison. 

Index Crime. The NYPD publishes the crime statistics it tracks as part of 
CompStat, focusing on seven categories of felonies — the so-called "seven majors" or "index 
crimes." These are (in order of seriousness) murder, rape, robbery, felony assault, burglary, 
grand larceny, and grand larceny auto. The CompStat index crimes differ from the Federal 
Bureau of Investigation's Uniform Crime Report ("UCR") system because CompStat 
classifications (a) follow the New York Penal Law's definition of the crimes (as opposed to the 
standardized UCR definitions) and (b) do not include arson as an index crime (arson is an index 
crime in the UCR). Not all felonies are index crimes — for example, although bribery and 
kidnapping are felonies under the Penal Law, they are not index crimes. 

Complainant. A person making a complaint to the police. Complainants 
typically come to police attention through 911 calls, by walking into a precinct, or by 
encountering a police officer on the street. 



Ray Rivera and Al Baker, Data Elusive on Low-Level Crime in New York City, N.Y. TIMES, 
November 1, 2010. In December 2010, the NYPD began publishing three categories of 
misdemeanors in addition to the seven major index crimes: petite larceny, misdemeanor assault, 
and misdemeanor sex crimes. The NYPD added this information to its public reports in response 
to accusations that the associated felonies were being downgraded to misdemeanors. 
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Complaint Report. The NYPD takes complaints in the first instance on a paper 
form — the Complaint Report Worksheet — traditionally known interchangeably as a "61" or a 
"scratch copy." Complaint reports are completed if the complainant describes a crime or has lost 
property. The information from the scratch copy is entered into the NYPD's Omniform system 
after which the operative complaint report is the Omniform version. 

Omniform. Omniform is a web-based software application that facilitates 
complaint and arrest processing at the precinct level. Omniform allows users to record 
complaints and/or arrest reports and stores those records for review, follow-up, and analysis. 
The system automatically generates complaint and arrest numbers. Complaint-report data from 
Omniform is utilized for purposes of CompStat. 

Downgrading and Suppression. There are two ways of artificially decreasing 
rates of reported index crimes: (1) downgrading and (2) suppression of complaint reports. 
Downgrading is classifying conduct that should be an index felony as a lesser degree crime in 
order to eliminate it as a reported index crime or reduce its apparent seriousness: e.g., classifying 
a felony assault as a misdemeanor assault results in an index crime being underreported. 
Suppression is preventing the reporting of a crime or failing to take a complaint report from a 
complainant. 

Uniform Crime Reporting. A program administered by the FBI, which collects, 
publishes, and archives crime-reporting statistics. The FBI provides contributing law- 
enforcement agencies with guidelines concerning how to classify offenses using uniform crime 
definitions — not the definitions used under each state's law. The data compiled by the FBI is 
utilized by law-enforcement officials, academics, the media, and others for research and planning 
purposes. 134 



See A Word About UCR Data, available at http://www.fbi.gov/about-us/cjis/ucr/word. 
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